Identity performance in the smart home

flatsiedatsie commented 5 years ago

During my research into existing smart home systems I realised that most of them are built on a certain technologist mindset.

Firstly, this way of thinking is connected to scientific ideals. For example, it is assumed that, just like in a science experiment, smart homes should register truthful data.

Secondly, this mindset has a tendency to recreate technological forms of power and apply their models. For example, it is assumed that the hierarchical models where a 'master' device controls 'slave' devices, or where an 'admin' has control over other users, should suffice when modelling how families interact in homes.

However, I believe the smart home of the future will never be successful if this mindset isn't augmented with knowledge and ways of seeing from the humanities. Already research is pointing out how smart homes are acting like top-down DIY surveillance systems that are creating unwanted social pressure and frictions in families, as family members actions become more transparent and trace-able to each other.

Let's start with an example. Imagine a teenager that wants to have some friends over while the parents are away for the weekend. Doing something sneaky like this is a normal and formative experience for many of us. Since smart homes are also surveillance machines, it becomes very difficult for a child to have this rebellious experience and get away with it.

This example points to two concrete problems with the current smart home ideology.

1. There is a hierarchical power structure built into the software that is inflexible, and incompatible with the complex power relations in families. The parent(s) fully controls the software. An admin user account can see all the data all the time, even from sensors or actuators in children's rooms.

What needs to happed here is that the smart home software allows for different, more complex forms of power relation development. For example, imagine an interface that allows a one-time creation of a child account where the admin might still be able to see some data in some situations (for example when the child's bedroom door is open), but not in others. The system would give the child user the power to create their own rules about how their data flows in the home. This would be irrevocable and not overridable by the admin. Another version would allow for a child account whose decisions are only overridable if the majority of users in the home force it. For example, in case of an emergency the door of the child's room could be forced to unlock if the parents and siblings agree that this is vital. In this example the power structure in the software mirrors a democratic system.

The point is not that smart homes should be democratic. The point is that smart homes should allow families to navigate these issues together, and create power relations that they themselves feel are balanced and acceptable. Currently smart home systems don't allow for this type of dynamic at all.

2. I realise this feels very counter intuitive to current ways of thinking about the smart home, but an ethical smart home should be able to generate fake data. For example, a child should be able to get 'the house on his/her side for a weekend', in order to secretly throw that party with friends (or to organise a get together with activists without it being registered, etc). Not collecting the data for a while is not enough, as that would also raise suspicion. The smart home system should be able to 'photoshop' the data from the weekend to look like a normal weekend. Here the smart home should act like the cool uncle instead of an unwavering snitch.

Systems like this are already being implemented. For example, some systems that collect data in cars have an ability to omit certain trips. This is used to avoid the bad driving behaviour of a friend to influence the carefully polished insurance score. Or just to be able to take a trip to a doctor/hooker/etc that is not registered.

The point is that people have all kinds of valid reasons to want to manipulate the data from these systems. And smart home systems should support our need to lie or omit things once in a while.

So what does this mean in practise?

First of all there is a huge opportunity for the Mozilla gateway here. Surpassing the competition is not about supporting more devices. That's the comfortable blind spot most technologists in this field end up in. Instead the Mozilla Gateway could create a unique selling point by developing a sensitivity to these types of real world issues on ethics and power relations and creating innovative interfaces that help family members distribute and limit the power to surveil, and to create ways to 'photoshop' data once in a while.

The reason my mom hesitates to buy a smart home system is not "it doesn't support all my devices". She doesn't have any yet. No, the reason is that she worries about these types of issues. Privacy, identity, control - or 'looking bad' and 'always getting caught'.

In practise I suggest the WOT standard and the gateway should reflect a sensitivity to these social complexities. For example:

To allow data to be 'photoshopped' the standard should support timestamps on data. It should be possible for devices to send data 'from the past'. And the gateway should support that old data could be overwritten/overhidden with new data. (perhaps a third party everyone trusts, like grandma, could still see the real data)

To allow for more complex power relations in families the WOT standard might incorporate some aspects of user control: who does the owner of the data want to allow to see the data? A few basic constructions could be created. And the standard could support that devices can have partial ownership for some roles and full ownership for others. A smart plug should behave differently as soon as it becomes clear that it has moved to a new room. It should be able to ask the gateway's "social power sharing" to force a vote by all family members on who owns the device now, and who can override control in which situations, or in which percentage of the time (e.g. "for two weekends a year I allow a user with the role 'child' to fake my output").

As a side-note it should be possible to tag some data as "should never leave the room/local network", and the gateway should store that preference and act accordingly.

In short, the smart homes and the standards that support them should be sensitive to human frailty, vanity, trust, imperfection, playfulness and power relations.

Remember: the smart phone became vital to our lives not just by being functional (who still calls nowadays?), but by becoming a tool for identity formation and expression. The killer app of the smartphone were the SMS and the selfie, as they offered us ways to present ourself to others in a way that made us look good. The smart phone was able to resonate with a core human need to project and sculpt a socially acceptable image.

Smart homes won't take off until they grow beyond being "routers for stuff" that think its normal to send behaviour data to third parties. Smart homes should start to support how normal humans perform identity. Which includes allowing them to lie about how often they exercised, what time they went to bed, and so forth.

After all, privacy is much more than 'the ability to control which bytes go where'. Privacy is the right to be imperfect.

benfrancis commented 5 years ago

Thanks for sharing your perspective, I agree with the overall point you're making.

Many of these issues are orthogonal to the technical specification you've commented on (although it's possible there may be a few assumptions baked in), but possibly more relevant to the design of Mozilla's gateway user interface implementation.

Do you have any concrete suggestions for features you think our implementation should have, or ways that it should be designed or implemented differently? I'm not sure how I feel about building in features specifically for people to generate fake data, but I certainly think users should have agency over their smart home data in general, have the ability to stop the data collection when they want to and control of what data gets shared with others.

flatsiedatsie commented 5 years ago

I suspect you're right. I realise WOT is about defining device capabilities and that a few of the things I propose are more about defining data flows, which can be said to be on a 'higher layer'. I can't really asses if there would be merit in pushing for a description of these concepts of complex ownership and 'social power brokerage' to a lower level. But it feels like devices of the future could pro-actively broadcast that they support these features on a device level. In the design of the smart home products I'm working on I could imagine something like this.

This is all speculative design, as I don't know of any system that explicitly tries to do these things yet, but I can imagine features like this:

A. "The family table boardgame - aka user management”

On this page in the UX new users can be created. Trying to think-out-loud on how family life could be modeled more complexely, I imagine this:

New users can be given rights that can not be taken away on a whim by the admin later. These are like digital human rights, in a way.
They are also given roles (can be multiple at the same time), and these can be added or removed.
Finally, they can have multiple superpowers (or 'responsabilites'), and these can be negotiated by the users together (more on that later).

Rights that users can have could be:

Be naughty. This allows the user to hide or massage data. The degree to which can grow as users (children) grow more mature and responsible. A teenager will need more 'rule bending' / 'cloak of invisibility' capability.
Own space. This is the right to have much more control over things in a space that is considered your own private space within a house. A child will want to have the ability to always have final say over a security camera in their room, even though everyone in the family benefits from its ability to catch thieves.
Cloak of invisibility. Camera's in the home never record data about this person. When they are recognised in footage, a machine learning algorithms wipes/massages the footage if they are the only ones in it. If others are in it, they are turned into a black silhouette.
Voting rights. This is the right to vote on things. It could be given binary (citizen or not), or with certain levels (can call a vote on phishing annoying behaviour of a fellow user (perhaps the sister is playing with the smart lights while he is studying), but not on security camera abuse). It might be that you want voting rights to be handed out differently per superpower.

Roles that these users can have would be based on the social positions that we find in most cultures:

installer. This is a power that is used by people that install the product in homes/offices, etc. It has a time limit, after which it 'dies'. It can be resurrected. It can change settings, it can see if devices are working/faulty/electrically powered, but it cannot see the actual data/state of devices.
owner. The person that bought the device has some unique rights from the very beginning. After all, they should be able to take it away to a new home and start fresh. The owner cannot see all the data, but can for example do a factory reset, do OTA upgrades for the gateway and devices they are also the owner of, and create a new installer account.
parent (rent payers). To delete a parent account all other parent accounts must concur. They have the ability to hand out rights to children and friends. Parents can also have access to some sensors that might be ‘jarring’ to children, like if the smart mouse trap is full.
child.
family member. Can get functions such as ‘arbiter’ or ‘surveillance godmother’, which can see the un-smoothed-over data.
close friend. Visiting guests are king - the are for example able to disable the recording of their presence via cloud connected voice systems.
friend. Being able to disable the recording via cloud connected voice systems.
guest. Being able to disable the recording via cloud connected voice systems. this account is automatically active when an unknown visitor is in the house, like a mailman or a plumber. - sleepover guest. An AirBNB guest who stays the night could need some special powers.
everyone. A role that everyone has. Everyone can turn of an annoying alarm after it has been ringing for a minute.

Superpowers are given to members of the household, and could be given to parents but also to children. They could be:

Marauders map reading. People with this function can find out there people and things are. This is a good example of a power that should be revokable/temporarily retracted if abuse is detected by the other family members. 
James Bond. This is the power to manage security devices. 
Cook. This is the power to use smart dangerous kitchen appliances.
Danger mouse. This is the ability to use dangerous devices, like lawnmowers. 
DJ. Has the ability to turn the music up really loud. Airbnb guests are not granted this power. 
Do not disturb. This could be a power to not be intruded or interrupted for a whole, for example to study or have sex. When this power is activated the room that this person is in cannot have it’s lighting intensity changed or sound device settings altered, for example.
Etc

Superpowers and Rights could be integrated, where things start off as a superpower (which can still be revoked, as a test phase to see if users are responsible enough), and later become rights when have proven themselves to be mature. For example, the ability to use kitchen appliances can be a power at first, and if the children don't make a mess, it can become a right.

Remember, with great power comes great responsibility.

Each year users can also be given “veto credits”, which they can use in special occasions to override a democratic decision. For example, whether or not to give a little brother the DJ superpower. Parents can choose whether to voluntarily submit to this system. It’s not possible to veto a veto.

Users could also be given “sudo credits”, which could be used in the example where a child wants to host a houseparty without anyone noticing, or wants to sneak out of the house to go on a date without it being registered.

The interface

What would the interface to manage this look like? I don’t know. The image of a boardgame keeps popping up. That is because these roles and powers should not just be granted top-down, but should also be negotiable. Democratically for example. Perhaps to have the DJ power you need 66% of people in the house to agree. Even visiting friends might be given the right to vote on that, while they are in the house.

The point of all this: user management has to be way more complex to better be able to reflect complex family relations, and to facilitate other types of power relations than "jailor oversees the jailed".

With all this is should also be kept in mind that this should not be a way to avoid talking to each other face to face about who is allowed what. The rights, roles and powers should be the result of dinner table discussion about who is allowed what.

.

B. Data photoshop

This would be easier to make. When users have the ability to photoshop data, they are allowed access to an interface where they can say:

copy the data from.. DATE&TIME A until DATE&TIME B

over .. DATE&TIME C until DATE&TIME D

for.. [ ] Actuator A [ ] Actuator B [ ] Sensor C [ ] Sensor D

The system would do some smoothing at the edges of those timeframes. For example, a humidity difference would not suddenly jump from 33% to 51%, but would slope towards that amount.

The user would be able do select many sensors to do this to at once. For example, to hide a party you would need to mask the data from smoke detectors, licht switches, (fridge)doors opening and closing, and so forth. At other times a child may just want to mask that they snuck out the back door to go on a date.

Of course this doesn’t hide all evidence. You’re still back at the same place kids are now: the parents can smell the smoke or detect that furniture is not exactly where it was before. But at least this way you have a sporting chance.

.

C. The App store

In order for a smart home to be more in tune with playfulness it will need to copy the smart phone’s ability to install apps. This is not just to install useful features. Quite the opposite: the smart home will become more fun and relevant if you want install silly apps, and fun games.

Remember the start of the Apple App Store? There were apps like “buy a jewel for 1000 dollars”, which were utterly useless but were great as a social signalling tool. People like to show off their wealth to impress others, and the smart home needs apps that allow this to happen.

Here’s some examples to give a broad idea:

The “Disco Dolly” app “Ooh, you got the Dolly Disco app!”. This app turns your living room into a disco by pulsating all the lamps and changing their color where possible. After all, why would you want to create a complex ’scene’ that does this when you can simply buy an app that does this for you?
The “Punk-d” app This app allow you to play pranks on your parents. It tries to lure parents to certain places in the home, for example by making them think the smart fridge is still open when it’s not. You can then hit them with a full blast from your watergun when they come down the stairs.
The “ghost app” This app makes it seem like there is another person in the home. This virtual persona ‘moves’ through the home in a realistic manner, making lights flickr and sensors beep as it passes. If your stereo is integrated it will also play haunting sounds, or randomly switch the TV channel when movies and TV shows about ghosts are on. Great for Halloween.
The “Casa Cluedo” escaperoom app Invite your friends for a fun evening or adventure. They must answer the voice assistants’ riddles and manipulate the sensors to progress through the home, discover who murdered misses Plum, and finally unlock the front door. But be careful - the killer still lurks around! Special devices - like a little lockbox with a keypad where you can hide a clue - are sold separately.
The “Crazy Frog home sound board” app Whenever you open the door of switch on a light the nearby speaker system plays a funny cartoony sound effect. Tionk! Whop!

Of course some apps will be useful:

The “I love you” app. This app automatically orders flowers to the home 3 times a year, on random dates (surprise yourself) or at dates set by you (valentine, etc). The delivery man is automatically allowed to open the front door when (for example) the smart doorbell camera detects a QR code or flowers (using machine learning). The hallway is open, but the rest of the house is still secure.
The “Business time” app This app turns any room into a workspace. It changes the lighting colour (if possible), and politely asks devices in nearby rooms to “keep it down”.

flatsiedatsie commented 5 years ago

WebThingsIO / api

Identity performance in the smart home #111