Sony Patents PS SkEYEnet To Watch Your Every Move, Emotion

It’s time to grab your tinfoil hats and prepare for the coming of an Orwellian nightmare – a new Sony patent highlights the possibility of a console that will track every aspect of your personality and life.

In a 2010 patent from Sony Computer Entertainment that has only just been published, Sony has detailed the creepy potential of a biometric controller and next gen PS Eye combination. The patent, innocuously titled “Automated Video Game Rating”, the company first describes how manual rating systems are flawed:

To guide consumers in their purchase, various rating schemes have been devised. Typically, these involve affirmative questioning of the users, and the use of their explicit input. Unfortunately, such an approach inevitably imports the biases of the users, and can lead to inaccurate or irrelevant ratings, because not all users are equally situated, and because haste, and sometimes deliberate dishonesty, factor into the rating session.

Fair enough – people do generally only rate a game if they really enjoy it or hate it, and there’s no way of knowing if that person has similar tastes to you. Sony’s proposed solution? To film you, watch you and listen to you.

The patent says that:

[A] Camera operates to observe the player during play, triggered to turn on when game play commences for instance. Facial recognition software, provided in camera or console , or even on a remote server such as rating server , serves to register various facial expressions and other behaviors of the player, in real time as the game is played. Expressions that can be registered include grins, grimaces, frowns, face contortions, looks of anger or astonishment, and the like. Each such expression is assigned a value that indicates a positive or negative experience and the degree of that experience at a particular point in game play. The values can be sent to ratings server through the network or stored on the console for sending at a later time when the conditions of network are more appropriate or the game deems more appropriate.


Other observational player data can also be collected. For instance, a sound recorder, integral with camera and/or console, can be used for voice recognition, to determine when expletives are used or when sounds or tones of contentment, triumph, anger or excitement are uttered. These incidents, measured in real time commensurately with game play, are also assigned various values indicative of their positive or negative nature and their degree. A user may shout “Hooray!” triumphantly upon defeating a foe in the game, for instance, and this can be detected and assigned a high positive value. Alternatively, the user can shout an expletive, which, if under expected circumstances, such as upon losing a life, may be assigned one negative value, while if under unexpected circumstances, as in uneventful game play, may be assigned a higher negative value as it may indicate boredom or lack of challenge or excitement. Other observational player data can include body position and posture and changes therein, along with various physical actions, such jumping up suddenly, flinging motions, and so forth.

So, at this stage, the patent allows the camera to watch your emotions, as well as listen to you. The level of accuracy of the watching camera could be potentially incredible:

  • Temperature of the player’s hands or face or other body parts (via an infrared camera for instance) as signs of stress, frustration or anger
  • Player’s gaze time for how bored they are to what is displayed on screen

Could the next PS Eye – which has been repeatedly patented – be even more precise than the recently revealed Leap camera (which is apparently 100x as precise as the Kinect)?

But the camera wouldn’t be the only thing used to work out exactly what you’re thinking. The patent mentions biometric controllers that measure your heartbeat and even sweat levels. This ties in to previous patents and rumors that the PS4 would have biometric controllers.

Using both these input methods, the console would also work out more personal aspects of your being, such as:

  • Hair style recognized (large, long, afro, bald)
  • Hair color
  • Facial hair recognized
  • Wears glasses
  • Color of clothing
  • Has an eye patch
  • Wears earrings
  • Wears facial jewelry (nose rings, etc)

This aspect of the patent is all the more baffling: while heartrates and sweat levels could potentially show your enjoyment levels of a game, what would your hair color have to do with it? Does Sony think people with blond hair like certain types of games, or that those with nose rings won’t like first person shooters?

In fact, the patent says something that isn’t exactly very politically correct:

Geographic information can even be used to point to cultural differences in players.

To get this information, the system could end up “analyzing IP addresses or by using GPS coordinates.”

You won’t be able to escape it when you travel either, with the patent saying that “the system can take the form of a portable gaming device”. If you have a 3G gaming device, it will then track where you play, and somehow work out your surroundings:

The type of games that are played while riding busses, trains or cars, or waiting in lines at a bank or at a bus stop can be tracked, based on geographic data, which can vary while the game is played. Also, indoor vs. outdoor game playing can be tracked, either geographically or using appropriate sensors, such as natural/sunlight detectors, provided for example on a portable gaming device. The geographical inference can even be resolve to the particular type of indoor room in a house or mall, or whether the player is seated at a desk, standing up in front of a TV, or reclining on a couch. Such information can elucidate player behavior and help match players to each other.

Don’t expect to be able to hide much from the system either if you have a Twitter or Facebook account, as it can access “databases such as those of social networks” to get your age and gender.

Of course, all of this is simply a patent and might never see the light of day, but even the fact that Sony could be contemptating such an invasive move sends chills down my spine.

What do you think, would you be comfortable with your every comment, every facial expression and every heartbeat being watched, recorded and analyzed? Let us know in the comments below.