Fingertip sensor gives robot unprecedented dexterity
Researchers at MIT and Northeastern University have equipped a robot with a novel tactile sensor that lets it grasp a USB cable draped freely over a hook and insert it into a USB port.
“People have been trying to do this for a long time,” Platt says, “and they haven’t succeeded because the sensors they’re using aren’t accurate enough and don’t have enough information to localize the pose of the object that they’re holding.”
Whereas most tactile sensors use mechanical measurements to gauge mechanical forces, GelSight uses optics and computer-vision algorithms.
“I got interested in touch because I had children,” Adelson says. “I expected to be fascinated by watching how they used their visual systems, but I was actually more fascinated by how they used their fingers. But since I’m a vision guy, the most sensible thing, if you wanted to look at the signals coming into the finger, was to figure out a way to transform the mechanical, tactile signal into a visual signal — because if it’s an image, I know what to do with it.”
A GelSight sensor — both the original and the new, robot-mounted version — consists of a slab of transparent, synthetic rubber coated on one side with a metallic paint. The rubber conforms to any object it’s pressed against, and the metallic paint evens out the light-reflective properties of diverse materials, making it much easier to make precise optical measurements.
In the new device, the gel is mounted in a cubic plastic housing, with just the paint-covered face exposed. The four walls of the cube adjacent to the sensor face are translucent, and each conducts a different color of light — red, green, blue, or white — emitted by light-emitting diodes at the opposite end of the cube. When the gel is deformed, light bounces off of the metallic paint and is captured by a camera mounted on the same cube face as the diodes.
From the different intensities of the different-colored light, the algorithms developed by Adelson’s team can infer the three-dimensional structure of ridges or depressions of the surface against which the sensor is pressed.
Although there was a 3-millimeter variation in where the robot grasped the plug, it was still able to measure its position accurately enough to insert it into a USB port that tolerated only about a millimeter’s error. By that measure, even the lower-resolution, robot-mounted version of the GelSight sensor is about 100 times more sensitive than a human finger.
In Platt’s experiments, a Baxter robot from MIT spinout Rethink Robotics was equipped with a two-pincer gripper, one of whose pincers had a GelSight sensor on its tip. Using conventional computer-vision algorithms, the robot identified the dangling USB plug and attempted to grasp it. It then determined the position of the USB plug relative to its gripper from an embossed USB symbol. Although there was a 3-millimeter variation, in each of two dimensions, in where the robot grasped the plug, it was still able to insert it into a USB port that tolerated only about a millimeter’s error.
“Having a fast optical sensor to do this kind of touch sensing is a novel idea,” says Daniel Lee, a professor of electrical and systems engineering at the University of Pennsylvania and director of the GRASP robotics lab, “and I think the way that they’re doing it with such low-cost components — using just basically colored LEDs and a standard camera — is quite interesting.”
How GelSight fares against other approaches to tactile sensing will depend on “the application domain and what the price points are,” Lee says. “What Rui’s device has going for it is that it has very good spatial resolution. It’s able to see heights on the level of tens of microns. Compared to other devices in the domain that use things like barometers, the spatial resolution is very good.”
“As roboticists, we are always looking for new sensors,” Lee adds. “This is a promising prototype. It could be developed into practical device.”