Sensors

Let’s start with two beer bottles. The Heineken Ignite and Stronbow StartCap.


What do these bottles share in common? They are both bottles of beer that double as media devices and sensors. Each of them was engineered by an advertising agency as part of promoting the brand of beer. We might say the advertisers expanded the affective capacities of the bottle. Bottles of beer have always affected consumers. You pop the cap, you drink the beer and its affects your body and mood. It makes you feel different. Sometimes excited, a bit buzzy, other times mellow, sometimes morose. What these advertisers did though was engineer the bottle into an input/output, I/O, device that can store and transmit information. The idea of an I/O device is a useful metaphor for thinking about ‘transfer points’ between digital media systems and our lives, bodies and societies. I/O refers to ‘input/output’: any program, operation or device that transfers data into a computing system.  A transfer of data is an output from one device and an input into another.  Hence ‘I’ Input and ‘O’ output. The I/O devices convert sensory stimuli into a digital form.  For example, the keyboard translates the physical movement of the fingers into a series of digital commands in a software program. The mouse translates the fine motor skills of the hand into digital data that moves a cursor on a screen.

The bottle becomes more than a container for beer. Heineken Ignite bottle had in its base LEDs, a microprocessor, an accelerometer and a wireless transmitter. These devices sensed and transmitted information. The accelerometer and wireless transmitter worked as sensors that could stimulate the lights in the bottle to flash to the beat of the music and the movement of people in clubs. Heineken claimed the intention was to create a mobile media device that captured people’s attention without them having to engage with the screen of the smartphone. Sort of. I think the advertisers understood full well that if you give drunk people in a club a bottle that flashes they are highly likely to capture images and videos of it on their smartphones for sharing on social media. The bottle is a device then that prompts people to convert the sociality of the club into media content and data on social media platforms.

Strongbow’s Start Cap followed a similar logic. It was sold in specially-engineered bars. When you flipped the cap off the bottle, an RFID chip in the cap would trigger responses in the club. For instance, the cap might pop off and that might trigger confetti to drop from the ceiling, or a light show to happen, or a song to play. These bottles are I/O devices that sit at the touchpoint between digital media infrastructure and human bodies. They sense action in the club, respond to that action, and in doing so stimulate responses from humans. Marketers are experimenting with beer bottles in part because they are an object that is held by the human body in social and cultural situations.
Here, we can see advertisers approaching branding as not only a process of symbolic persuasion. They are not really here making an ‘advertisement’ that contains a message we consume, rather than are engineering a cultural experience. They are using media as a data-processing infrastructure to sense, process and modulate humans, their feelings, bodies and cultural practices

We should pay attention to advertisers in part because they are critical actors in experimenting with new media technologies. Via branding exercises like the Heineken Ignite and Strongbow StartCap we can see advertisers treating media as data-processing sensors. Jeremy Packer suggests that the capacity to exercise control using digital media is ‘founded upon the ability to capture, measure, and experiment with reality’. In the present moment, we need to pay on to the increasing capacity of media to ‘sense’, calculate and experiment with our lived experience.

These two beer bottles are part of a larger process of weaving digital media and networks into our everyday infrastructure. This gets called the ‘internet of things’. Watches, Televisions, Cars, Fridges, Kettles, Air Conditioners, Home Stereos are just some of the everyday objects that are getting ‘connected to the internet’. My friend’s dog is even connected to the internet. Well not the dog itself, but the dog’s collar. They can load an app and see where the dog is while they’re at work. This ‘thingification’ of the internet is promoted to us as living in sensor-rich smart homes and environments.

As you drive home, your car knows when you are getting close and turns on the air conditioning, and maybe flicks on the kettle. You can think about how the logic of turning everyday objects into sensory devices works. Once your car is a sensor, it can start collecting all kinds of information. Say there is a sensor in the steering wheel that can record information about how erratically you are driving, or say there’s a microphone in the car that can hear the tone of your voice. The car might be able to sense what kind of mood you are in as you drive home from work. In a bad mood? It might tell your home stereo to put on some chilled out music and dim the lighting by the time you arrive home. OK, I kind of made that up. But, it’s not ridiculous.

Platforms like Google and Amazon imagine us living alongside all sorts of artificially-intelligent things. You open the fridge and say ‘Ah, we’re out of milk!’ Your home assistant hears you say this, and puts it on your shopping list. If you get home deliveries it might automatically order it for you. If not, it might sense when you are at the shops and send a reminder to your phone. A basic point I’m trying to draw out here is that the engineering logic of media platforms does not begin and end with the smartphone and its apps. Platform engineers consider that all kinds of everyday objects will be ‘input/output’ devices that are incorporated within the platform architecture. These devices act as ‘switches’ or ‘transfer’ points between the bodily capacities of consumers and the calculative capacities of media platforms. These devices sense by recording information about the expressions and movements of humans and their environments, they translate by transforming reality into data that can be processed, and they stimulate by delivering impulses and messages to users. I think of these devices as ‘affect switches’ in the sense that they transfer the human capacity to affect into the calculative apparatus of media infrastructure. A device that can ‘sense’ your mood by recording your voice, or your movement, or what you’ve been tapping or swiping for instance is translating some information about your lived experience – how you feel ¬– into digital data. And then, processing that information and making a decision about how it might modulate your mood.

To affect is to have influence on or make a difference to, this is often particularly meant in relation to feelings or emotions. A switch is a device that can coordinate or stimulate movement in a system, it can turn something on or off, or change its direction or focus. An ‘affect switch’ then is a device that can alter the direction of human action or attention. Affect switches are techno-cultural devices for conducting and governing the dynamic and indeterminate interactions between consumers, material cultural spaces and media platforms. The beer bottles I started out with are affect switches. They sit at the touchpoint between body and media platforms. They sense information in the environment, and then stimulate particular moods and reactions from users.

OK, there’s another crucial point these beer bottles help us make. Popular culture can sometimes seduce us into thinking new media is about virtual simulations out there in cyberspace, that media is somehow ephemeral. That’s a ruse, digital media are material objects and infrastructure. They exist in the real world, and involve the transformation of real world objects and spaces. The beer bottles are one example of everyday objects ‘becoming digital’. They retain their material character and place in our world, the change is that they are now connected to a digital media infrastructure. Mark Andrejevic and Mark Burdon suggest that this world where more and more objects become touchpoints between our lived experience and the data-processing power of digital media is a ‘sensor society’. They suggest our homes, workplaces, cars, shopping centres, and public places are filling up with 'probes [or sensors] that capture the rhythms of the daily lives of persons, things, environments, and their interactions'.

In their way of thinking a sensor is 'any device that automatically captures and records data that can then be transmitted, stored, and analysed' they 'do not watch and listen so much as they detect and record'. This leads them to make a really critical point. When we see a device as a sensor in a sensor society we must think not only of what it records but also how it is stored, who has access to it and how it is used. We are all ‘sensed’ by sensors, we all have data collected about our bodies, movements and expressions. But, who gets to do the sensing? Who gets to keep, process and benefit from all this sensory information that is collected? We live in a world where more and more everyday objects are becoming sensors that collect data about us.

This prompts us to rethink the ways in which we participate in a digital world. Much of our participation is relatively passive. Passive data is the kind of data that is collected through sensors, it is data that we do not necessarily consciously know we are creating. Sure, we might immediately think of our smartphone here. Often times it is collecting data that we don’t really think about. Go check your location services on your phone. Unless you switched it off you’ll see it has a fairly complete record of where you go. It’s probably identified your home and work.
Periodically there is controversy about apps using the microphone to passively monitor your conversations. Here are moments where we are not actively participating by using the phone to say post something to social media, rather it is passively sitting in the background monitoring us. This kind of passive monitoring goes way beyond the phone.

Here’s two examples, one kooky, one creepy.

Kooky first. In July 2017, it was reported that Roomba vacuum cleaners were collecting information about your home. The vacuum needs to collect data in order to learn how to the vacuum your home – to figure out where walls and furniture are. It creates a map of your home. But, it doesn’t just use that map for its own cleaning. That map is also a data set about what objects it ‘bumps into’ in your home. The data is stored by the parent company. They are considering selling it. The data could be used to make predictions about what kind of family you have or what kinds of objects you own. And, from there, advertising might be targeted accordingly.

OK, and here’s creepy. Earlier in 2017 the ‘smart’ vibrator manufacturer Standard Innovation settled a lawsuit for $3.75 million. These vibrators allowed users to remotely turn on their lover using a Bluetooth connection. Two hackers demonstrated how the vibrator could be hacked and remotely activated. But, get this, the smartphone app that was used to control the vibrator collected information about users, including information about temperature and vibration intensity without users consent. So, here it is, an intimate personal object doubling as a sensor that transfers information about sexual practices back to unknown third parties.

For Andrejevic and Burdon the sensor society is not just a ‘world in which the interactive devices and applications that populate the digital information environment come to double as sensors’ but also the emerging practices ‘of data collection and use that complicate and reconfigure received categories of privacy, surveillance, and sense-making’. The users and collectors of the troves of data that sensors collect range from government spy organisations such as the NSA, to data analytics companies, to advertising companies, insurance agencies, hedge fund managers and the companies that collect the information in the first place ranging from social media platforms to appliance manufacturers like General Electric. The organisations that can access this sort of big data are not ordinary individuals.  By its very nature this data is useful only to entities that want to measure and affect large numbers of people – those who want to act on a society wide level.  
Andrejevic and Burdon tell us that ‘structural asymmetries (are) built into the very notion of a sensor society insofar as the forms of actionable information it generates are shaped and controlled by those who have access to the sensing and analytical infrastructure.’

A sensor society then is one where everyday objects are connected to a digital media system. These objects collect data. The consequence of having more objects, in more everyday situations collecting more data, is that we are becoming a society characterised by the collection and processing of information on an enormous scale. As we become a society that collects more data than any humans can interpret, we begin to create machines that process that data and make decisions. Patterns of human life that are not visible to humans, are visible to machines. A sensor-driven media system doesn’t care for what we think or enabling us to understand one another as much as it aims to develop thecapacity to make us visible and to predict our actions. 'Machines do not attempt to understand content in the way a human reader might'. A human would be unable to keep up with the vast amount of data involved but algorithms and artificial intelligence can. Sensors are a critical part of the larger media platform eco-system. Sensors ‘connect’ that system to lived experience and living bodies, they enable calculative media platforms to learn about human life, and as a consequence, make more machine-driven interventions in it.