Inside Project Natal's Brain From: Popular Science - 02/2010 - page 4 By: Jill Duffy The artificial intelligence behind Microsoft's Xbox 360 motion-sensing game controller Project Natal is Xbox 360 controller-free gaming add-on that recognizes even the smallest human gestures. Project Natal lets you control a game just with your body movements—no buttons or Wii-like wands—by watching you with a 3-D video camera. It's the software inside, which Microsoft casually refers to as “the brain,” that makes sense of the images captured by the camera. In programming this brain - a process that's still going on - Microsoft relies on an advancing field of artificial intelligence called machine learning. The premise is this: Feed the computer enough data—in this case, millions of images of people—and it can learn for itself how to understand it. That saves programmers the near-impossible task of coding rules that describe all the zillions of possible movements a body can make. The process is a lot like a parent pointing to many different people's hands and saying "hand," until a baby gradually figures out what hands looks like, how they can move, and that, for instance, they don't vanish into thin air when they're momentarily out of sight. Microsoft is currently training and improving the version of the brain that will ultimately go into the final product by painstakingly gathering pictures of people in many different poses, and then running all this data through huge clusters of computers where the learning brain resides. The learning brain, shown is a complicated cluster of computers. What's the brain thinking as it watches you jump around, swinging imaginary bats or head-butting imaginary soccer balls? As you stand in front of the camera, it judges the distance to different points on your body. Then the brain guesses which parts of your body are which. Once Natal has determined it has enough certainty about enough body parts to pick the most probable skeletal structure, it outputs that shape to a simplified 3D avatar. Then it does this all over again—30 times a second! As you move, the brain generates all possible skeletal structures at each frame, eventually deciding on, and outputting, the one that is most probable. This thought process takes just a few milliseconds, so there's plenty of time for the Xbox to take the info and use it to control the game. Microsoft is not divulging details about this hardware yet, but we do know that it measures relative distances using a black-and-white camera sensor and an near-infrared beam. Read the entire article at: http://www.popsci.com/gadgets/article/2010-01/exclusive-inside-microsofts-project-natal Gallery: http://www.popsci.com/gadgets/gallery/2010-01/natal-gallery Project Natal http://www.xbox.com/en-US/live/projectnatal/ Project Natal YouTube http://www.youtube.com/watch?v=g_txF7iETX0