If your friend says she feels relaxed, but you see that her fists are clenched, you may doubt her sincerity. Robots, on the other hand, might take her word for it. Body language says a lot, but even with advances in computer vision and facial recognition technology, robots struggle to notice subtle body movement and can miss important social cues as a result. Researchers at Carnegie Mellon University developed a body-tracking system that might help solve this problem. Called OpenPose, the system can track body movement, including hands and face, in real time. It uses computer vision and machine learning to process video frames, and can even keep track of multiple people simultaneously. This capability could ease human-robot interactions and pave the way for more interactive virtual and augmented reality as well as intuitive user interfaces.One notable feature of the OpenPose system is that it can track not only a person’s head, torso, and limbs but also individual fingers. To do that, the researchers used CMU’s Panoptic Studio, a dome lined with 500 cameras, where they captured body poses at a variety of angles and then used those images to build a data set.They then passed those images through what is called a keypoint detector to identify and label specific body parts. The software also learns to associate the body parts with individuals, so it knows, for example, that a particular person’s hand will always be close to his or her elbow. This makes it possible to track multiple people at once.