an evening with Kinect · 2010-11-10 23:44
I broke and bought a Kinect today. Well, it is the future of interaction and all that. An hour of digging out my Xbox and 3 system updates later, the Kinect whirred into life, the slightly scary red diffractor started to shine, it listened to the room and its own echo for a bit, and the character on screen assumed my position.
Kinect uses an infra-red speckle to determine depth
The initial calibration in Kinect Adventures is very simple – move as far back as possible so it can work out the size of the play area. The games themselves are very reminiscent of Wii Sports although there’s a little more movement and jumping involved. The first person shadow you perform through is a bit distracting but you get used to controlling it.
It was only when I loaded Dance Central that the reality hits – it’s really monitoring your body completely, and has some idea of a body and how a body moves. It’s disconcerting. There’s no cheating and no micro-moves that work like on the Wii. Describing how to correctly carry out the moves is even harder than on the Wii as there’s no object to relate to – there’s so much that you could be doing wrong. Dance Central highlights body parts in red if they’re not right, but it’s a lot of information to parse whilst still trying to keep in rhythm and get back on track.
As an aside, the navigation UI in Dance Central is one of the nicest and most satisfying I’ve used – rather than ‘hunt the circle and wave’ from the Wii and most Kinect games, your right hand selects with a swipe, and left hand for back and other alternatives. It also shows a small black and white picture-in-picture of what information it’s picking up from you and it shows that it can’t monitor some of the more extreme movements, such as putting your hands in the air (and waving them like you just don’t care).
There’s one ubiquitous command across all Kinect games and UIs – the equivalent of a G or 7 in semaphore. This pauses the game and brings up a menu of options. I can’t wait for the first music video dance sequence to add this in.
Your Shape, one of several fitness titles already available for Kinect, highlights even more the unobstructed space needed for Kinect to work (many British living rooms will not be big enough for it to work correctly, especially in two player mode) plus the importance of sensor placement. I’ve currently got it resting on the base of my TV but Your Shape really needs it to be mounted on top of the TV, which requires an additional mount. Currently the scary smooth blob representative figure Your Shape uses has its feet cut off.
When you go to the Kinect calibration settings, it shows a slightly different raw view from the Kinect depth sensor with a skeleton stick man drawn onto it to show where it thinks your head, body and limbs are.
Your Shape ups the unease by ‘scanning’ you, showing a complete skeleton over your body and then providing measurements – to the centimetre – of your legs, arms, torso, hips and waist. I’m not sure how accurate these are (and they’re only shown on screen for a second) but if this information can be inferred it could really revolutionise clothes shopping – both in store and online.
You can create a Kinect ID that automatically logs you in when you walk in front of the camera and wave. To create the ID, you perform a long series of directed stances – kind of like the original 3-pose airport body scanner dance but on a flashing light-up dancefloor. It doesn’t like sunlight but needs more light than most living rooms have to create and use the ID – and preferably needs scans in all different lighting conditions to work. It took 3 attempts (nearly half an hour) of posing for it to have enough information to create the ID and recognise me. Whilst these are stored locally, I see most of the benefit (and the scariness) when these are packaged up and stored and searched centrally – imagine just being able to walk in front of a friend’s console and it recognises you.
There are two coups in Kinect that people aren’t really talking about. The first is speech recognition – the (albeit limited) Kinect menu takes speech commands prefixed by a magic word (Xbox) and seems to work well. While the body tracking was interesting and slightly scary, there is nothing better than a computer responding to your speech. I don’t think any games currently mix movement and speech commands but there’s a lot to build on there.
The second is that you’ve basically bought a video conferencing system better than many commercial offerings costing more than 5 figures. I can’t wait for Skype to come to Xbox.
Kinect feels like first generation hardware – it works, just, most of the time. There is a definite lag in detection that games have to build in – frustrating when many games are based on rhythm. It’s really unclear how accurate the system is and I suspect many games have dialled down the accuracy to ensure reliability of detection. I suspect the lag also contributes to the extreme unease the system sometimes creates – we’re deep in the uncanny valley of interaction.
The playing field is a little farther from the TV than you might want – I kept on moving closer, even though I’ve got a 42” TV. You need a big living room and a bigger TV. The body detection is clever but can’t cope well when limbs are in front of each other – crossing your arms really confuses it. I suspect the next generation will need several cameras to make a true 3D representation – and that will mean another living room rearrangement.
comments
Begs the question of design and testing of physical UI in/for american living spaces vs japanese ones?
Thanks for this. On three occasions I’ve been one click away from ordering one online, but lag is the deal breaker as far as I’m concerned. The reviews I’ve seen for Dance Central (which seems to be the only decent game of the launch line up) didn’t flag it as an issue. I take it your comment on lag applies to Dance Central too? I’m wondering if Microsoft’s hands might be tied with respect to future system updates that could reduce lag if they’ll affect the core gameplay of games already released. Presumably developers are aware of, and have in some way responded to, the lag problem. I suppose concurrent patches for the individual games would be one option.
As things stand, the facial recognition and voice control are, to me, more exciting than the motion tracking. Maybe Child of Eden will convince me that the body-as-game-controller paradigm can work. Otherwise I wonder if limiting interactivity (or merely slowing games down) is the way forward? I hate to say it, but a game along the lines of Myst or 7th Guest might be interesting. Of course, if Project Milo had been a reality, I’d probably have ram-raided Currys.