MACHINE LEARNING CITY

Whether it's camouflage makeup to avoid facial recognition (CV-Dazzle, 2010) or a geolocation spoofing device (Skylift, 2016), which make every phone think that it's in the Ecuadorian embassy in London, Harvey's work revolves around the issues of surveillance, IT security and the future of biometric information in the context of data protection. Many of the American media artist's works use artistic interventions to mediate between invisible but powerful mechanisms and images and their influence on our daily lives.

For the 2018 Elevate Festival, Harvey dealt with precisely these contexts. In his commissioned work he creates a direct reference to Graz. What role do places and people in Graz play in the development of surveillance algorithms used by governments, security companies, and self-driven cars all over the world? How important is individual consent to the use of an image in the digital age - an age in which images lead a life of their own, operate outside our control and to feed invisible technologies that we use on a daily basis?
Harvey flips the perspective in his Machine Learning City piece- he looks back at the images with which we learn machines to ‘see‘. The images shift away from the academic sphere in which they are otherwise located - his topic is a research area at several universities - and thus merge into the art world. They open up a new aesthetic value to us, which has been called ‘New Aesthetic‘ in recent years - a term coined by the author and artist James Bridle in 2011 to give a name to an artistic movement whose technological appeal ushered in a new visual regime. The pictures are highly political because they depict how we program our future.

In the following interview with Elevate Arts curator Berit G. Gilma, he talks about the project in Graz, in which Harvey addresses the question of the right to one's own image in the context of urban discourses such as surveillance, public space, and digitization.

Berit Gilma: Tell us more about the project ‘Machine Learning City‘ that you have created for Elevate Arts 2018 and what role does the city of Graz play in it?


Adam Harvey: The project ‘Machine Learning City‘ explores the new reality of cities as datasets. But the often overlooked part of this reality is the story behind the data. How was it acquired and who might be in the dataset? ‘Machine Learning City‘ tells the story of a now famous pedestrian detection dataset, which actually originated in Graz in 2003. During the next decade, this dataset became the most widely used pedestrian detection dataset in the world. People from Vienna, Graz, as well as the neighboring city Leoben, play a unique and historic role in the ability for computers to see and understand people. 




Berit Gilma: It seems this kind of databases are collections that are mainly circulating in the academic sphere, invisible to the general public. But they are powerful images of our time: they train self-driving cars and surveillance technologies like person detection or face recognition algorithms used for example by security branches of governments. What issues do you see in these databases and how do you use art to communicate them to an audience?


Adam Harvey: These databases also circulate widely in the commercial sphere where they are directly used for training and evaluating surveillance software that is licensed to government agencies. Although this information is public it is not widely known. Citizens aren’t aware that when they go outside and walk down the street there is a possibility that they are walking right into someone’s research project. When people are unknowingly captured for research and then distributed to surveillance companies around there world, there is a clear breach of ethics and privacy. But if we step back from these concerns, the larger issue is what does it mean to be a city engaged with machine learning? Is the public a place where algorithms can learn? If so, can other countries extract data from your city? As cities move towards further integration with artificially intelligent infrastructures will there be any limits on visual data consent?




Now as we are in the post-Snowden era, where there is more awarness about governmental surveillance, how would you say did computer vision technology change the discourse on it?


Actually, I don’t think the Snowden documents changed discourse on computer vision. The problem is that we consider visual information and signals intelligence as two separate domains when technically they are all part of the same spectrum. The significance of recent breakthroughs in computer vision capabilities is not widely understood. And as people learned from the Snowden documents, those with the means to exploit unprotected communications will have massive advantage in gathering information. Computer vision will be the biggest signals intelligence exploit of this decade. 




Does computer vision change the way we navigate in the world? Or in other words, does computer vision technology influence our perception of the world?

If we reframe computer vision as a type of wireless data, then we can imagine computer vision will have as much impact as WiFi, which is to say it will have a significant impact on the way we navigate and interact with the world. Cities will have to make small adjustments to become more machine readable by autonomous vehicles, but I think the larger change will be how people become more or less machine readable. Ultimately, I think it will change the way we dress every day. This is a topic I addressed in an earlier project called CV Dazzle, a type of camouflage from computer vision. 



You recently exhibited a project called ‘Megapixel: Face Database Query‘ in the Glass Room* in London where people could see if their face was in a database that trained face recognition algorithms. How many people found their faces and how was their reaction?


At least 2 people reported finding their identity in the facial recognition training database used for MegaPixels. In total the database contained about 672.000 identities and 4.7 million images. The chance of anyone finding their identity was quite low, but the facial recognition software worked surprisingly well. The database, called MegaFace, used in the project is significant because it’s used by government contractors around the world (USA, China, Russia) to train and evaluate products they license to the security and defense industry. Several of the companies publicly post their score on the MegaFace dataset website. It must be an odd feeling to realize that a random image your posted years ago on Flickr has somehow influenced the capabilities of foreign security services to find enemy combatants. Eventually the MegaPixels software will be publicly available online for anyone to use. 



What is dataconsent? Where do you think is the line between the public and the private in the digital and physical sphere?


Even if a company obtains your consent, it’s often asymmetric. It’s just not possible to agree future uses of data that no one yet understands. Data consent will become a major issue as cities play host to international technology companies that require continued access to public data to ‘improve services‘. For example self-driving cars will need massive amounts of data to ensure they operate safely. To do so they will need to continuously collect and train on local data. This data may also include license plate numbers. But even if these are blurred, it’s already been proven possible to identify an automobile by the made, model, and year. In other words, you can do license plate level recognition without a license plate. This is just one of many examples where providing data consent becomes problematic. As the Strava* heat map illustrates so well, metadata can reveal a lot.



Are there potential negative outcomes if my face ends up in a database?


Right now you don’t have a choice. This is problematic because your face, which has become commoditized as a security token, could be spoofed or replicated elsewhere. It’s bad security practice to share the data that you’re also using for authentication. The other negative outcome is that current data collection practice becomes standardized or normalized to the point that you don’t actually own your face.



How much do you know about the legal situation in collecting images of person ’in the wild’?


The term ‘in the wild‘ was popularized in 2007 by the researchers who created the now famous ‘Labeled Faces in the Wild‘ dataset. This was the first time a face detection dataset had been created entirely online. In the past, researchers would have to invite people to a studio, capture their image, and acquire written consent. This also yielded unrealistic images. For computer vision algorithms to work well they need realistic data. Now this typically comes from social media sites, where users are unwittingly contributed towards building machine learning datasets that eventually train surveillance algorithms that analyze their social media images.



Images take on their own life in the digital age of the internet. In another work of yours you are using the slogan: ‘Todays selfie is tomorrow’s biometric profile‘. How do you see privacy concerns in the future with our selfie culture today? 


It’s true. Selfies are helping to create incredible large biometric databases that are used to train facial recognition software that is eventually licensed back to law enforcement and government agencies. As an experiment I built a selfie image scraper in 2016 and was able to obtain 36.000 unique selfies per day from Instagram alone. We need to rethink visual privacy and the consequences of publicly posting photos. A very small image can provide a large amount of information.


Do you put your face online? How do your privacy concerns affect your daily life?
I limit what’s posted online and will ask people to remove face images. I’ve asked photographers at events to delete my image so it’s not posted online. Part of the problem is a lack of well designed tools to remove biometric information. I think there will be many opportunities for privacy enhancing technology in the next few years.

* 'The Glass Room' was an exhibition in London from 25th of October to the 12th of November 2017. It gave an insight into the use of personal data in the digital age. The exhibition and the rest of the program was produced by Mozilla and curated by the Tactial Tech Collective.

* Strava is a fitness app that unintentionally revealed CIA's secret locations in January 2018. Employees used the app, whereupon their fitness data was published online. The resulting heatmaps in remote locations, for example, revealed locations in the area 51 - a history-steeped secret base in the US state of Nevada.

Machine Learning City
01.03. - 09.03.2018, täglich 14-19 Uhr
esc medienkunstlabor
Bürgergasse 5, 8010 Graz