It’s in the Eyes

Last week I participated in a game jam using eye-tracking technology. The jam was sponsored by Tobii, who supplied the EyeX eye tracker kits used. Historically, eye-tracking gaming has been decidedly niche. For example, being used by disabled gamers who have limited movement. Interestingly though, Tobii are looking to mass market the EyeX, with a particular focus on core gamers. The principle being that certain parts of a player’s game can be improved using the eye trackers.

The example Tobii highlighted was of aiming with the eyes in an FPS, but otherwise using normal mouse and keyboard controls to move around. Not having to physically move the mouse over an enemy, once identified, made players’ aim far faster and more accurate.

No Strategy

Taking that on-board for the jam, I decided to experiment with another core genre – strategy. The eye tracking technology itself is wonderfully responsive and integrated relatively easily with Unity. After some playing around, I decided to go for some sort of area control mechanic. Players would draw shapes on a map with their eyes to capture territory.

However, people’s eyes have a tendency to flit around the screen, often to things they wished to be inside or outside those shapes. Which of course, led to people drawing through the middle of those objects instead of around them.

One solution I explored involved a snap-to-grid based system. To place a shape boundary point, players just had to keep their gaze roughly inside a single square on the map for half a second. Unfortunately, it takes an uncomfortable level of mental effort and puts too much strain on the eye to artificially stare at a set point for any more than your subconscious wants.

Small Things

Another mistake was the timer. I already had a cursor to show where the player’s current eye gaze was. However, to give more feedback I added a radial fill timer around it, to show how long the player had left to gaze on a fixed point.

The problem with this approach was that the eye would start looking at different parts of the timer to see what progress it was making, thereby changing the position of the cursor. Locking the cursor position onto the centre of the grid square, and having the cursor snap from place to place felt unnatural. Furthermore, the eyes would see that the timer was just about done, and so flick to the next place of interest in anticipation of being able to lay down a new point or do whatever else, thus cancelling the current action at the last moment. To say this was incredibly frustrating would be an understatement.

The general idea of a cursor however, worked well. Almost all the other jam participants had some form of cursor or object in their games to show where the eyes were looking at any one time. Universally, those cursors were small. A little cross or dot wouldn’t distract or get in the way, but was still useful. In the first place, to let the player know where they were actually looking. Since the eye tracker technology needs calibrating, the player could see if it was slightly off, and so readjust the screen or their seating position.

Moreover, it’s disconcerting to have things react on screen with no visible cause or other in-game object to interact with. The cursor acting as a proxy for the player helped plug that gap.

Halve the problem

Much of what I was doing came down to completely swapping out a traditional mouse/keyboard or controller with eye tracking. As I discovered, trying to do both selection and interaction (clicking/pressing) at the same time isn’t possible with a single input. Many of the other developers doing the jam instead sidestepped the issue.

The game Island, simply took controls away from the player. From their God-like vantage point in the sky, players would gaze down over the slowly rotating eponymous island. With movement already taken care of them, their eyes were thus set free to plant trees or watch over the tiny citizens roaming around the island, (or whatever else Gods do in God-games). The slow pace of the rotation added to the chilled out atmosphere of the game.

Medusa’s Tempest was a twin-stick shooter that had players aim with their eyes, whilst moving their ship with more traditional controls. On the surface, it still had the problem of players having to look both where they are going, and where they are aiming at the same time (which is obviously impossible). In practice, once they’d played the game for a short period of time, players were able to remember that they’d pressed the stick in a certain direction for so long, and so gone so far. Thus could building up a rough mental map of where on screen they were at any one time.

Another feature of Medusa’s Tempest was to replace bullets with a constantly streaming laser. No need to scan around checking your bullets’ progression to their target. Equally, no accidentally firing your next bullet at the last bullet you just fired. Instead players can just concentrate on the targets.

Additionally, having a constant line from the player’s gaze point, back to their ship, assisted in keeping mental tabs on whereabouts their ship was, whilst not interfering with shooting.

To The Rhythm

Beat Shot, which took home top prize at the jam, in a similar vein to Island, also had a less-is-more approach to controls. Only this time, in the form of a rhythm game. On every beat of the music, the game would send out a pulse from the point where the player was looking. If any targets were within pulse range, the player would “hit” them. The game worked so well in large part due to the timing. There was just enough between beats to process the results of previous hits, glance between potential new targets, and then concentrate on one in time for the next beat to fire off.

The music was set to 120bpm, which gives you an idea of just how much the eye can take in and brain process in such a short space of time.

There are also aesthetic considerations as well. The maker of the game EyeBall, (think basketball, but aiming your passes and shots with your eyes), started off the game in first person, but changed to third person. Simply because it was a more satisfying way to watch the ball arc through the air.

Before the jam, we were told the best eye-tracking games were all about movement (something I really should have listened to!). I’d probably qualify that by saying the best put the player’s eye movement at their core. That could be directly moving things with the eyes, such as in Medusa’s Tempest. But also simply working with the way the eye moves around a static scene, as with Island and Beat Shot.

Finally, I’d like to give a shout out to Alice and the team at London Game Space / Scenario Bar for hosting the jam. Also for the charity Special Effect, who already do a lot of work with eye tracking in their quest to make games truly accessibly to everyone, and were able to come down to the jam on day one and give us some pointers and tips.