Disney Research Pittsburgh has just released the video below which demonstrates one of its latest projects: an audio animatronic robot that can interact with people by playing catch with them. The system uses an off-the-shelf Microsoft Kinect (according to the video’s narration) along with an external camera system (ASUS Xtion PRO LIVE) to locate balls and a Kalman ?lter to predict ball destination and timing. So not only is the robot able to track a human’s position and size by the location of their head, but it can attempt to move its hand to catch the ball. If the robot misses the catch, it’s fully aware and even responds with one of several different humorous animations to elicit a response from the person interacting with it.
Disney Research has also been able to use its system to successfully juggle up to three balls at a time when a professional juggler is used as the participant.
According to the video and its description, Disney Research is hopeful that this product leads to a fully interactive experience between guests and audio animatronics in environments such as theme parks, while managing to keep the guest a safe distance from the robots.
After presenting its technique for cloning the human face in an effort to produce more realistic audio animatronics at SIGGRAPH, Disney Research Zurich has released this video which takes a closer look at the process, which we began discussing on here last month.
The video starts off by giving an overview of the patent application which we previously described. Essentially the project aims to correct issues with traditional audio animatronics in which the synthetic skin is stretched as actuators contort it to form various expressions. By using an array of high definition cameras to produce marker-less motion capture, the system can accurately determine how a specific synthetic skin material, such as silicone, should be cut in terms of varying thicknesses and attached to the animatronic skeleton so that the desired expressions are replicated precisely, down to the wrinkle level. The video then goes on to give a full demonstration of the process, from scanning the subject, to producing the mold, to comparing the original actor with his audio animatronic counterpart.
In this day and age in which 3D scans of human faces are turned into exciting keepsakes such as the Disney/LucasFilm Star Wars Weekends experience ‘Carbon Freeze Me,’ in which guests could receive a replica of themselves frozen in carbonite a la Han Solo, and the upcoming ‘I Am A Princess,’ which builds on a previous test in which guests could have a princess doll in their likeness made, technology is becoming a key player in what has been even the most traditional of trades.
Therefore it shouldn’t come as a surprise that one of the pioneering technologies employed by The Walt Disney Company is being updated in a fascinating new way that will attempt to make audio animatronic figures rival the most advanced 3D, high definition screens. The ominous-sounding ‘Physical Face Cloning’ patent application (US 2012/0185218) seeks to improve upon the decades-old theme park experience by using some complicated algorithms to produce the most life-like audio animatronic figures to date.
Based on the listed location of the majority of the inventing team, the project appears to come out of Disney Research in Zurich, Switzerland. Disney Research recently came into the limelight with its technology dubbed touche, allowing users to control devices by gestures.
The problem with today’s audio animatronic figures, according to the patent, is that they require enlisting a team of animators, sculptors and other experts to create a face and skeletal system able to produce a realistic set of human expressions, as relayed through a layer of artificial skin. Taking the guess work out of the process, the new system could simply use motion capture technology to record the human subject’s face making various expressions and, via some very non-simple mathematical formulas, generate the perfect layer of silicone rubber skin (or whichever material is desired) of varying thickness, along with directions for attaching said skin to the skeleton, so that when the skin is stretched and manipulated on the figure to form the desired expressions, it provides the most realistic visuals possible.
UPDATE #1: 7/25/12 – Disney Research will discuss the new technology at SIGGRAPH 2012 on August 9. More information here.
Let’s face it, audio animatronic figures are just a passing fad. Sure they look and move in realistic fashions, even so much as being able to appear to walk steps or be able to twirl a lasso, but it all comes at a high cost — literally and figuratively. They’re expensive to design and create, require expensive maintenance and heavy and large platforms and have extremely limited mobility. And that’s coming from one of the leaders in robotic development at Disney Research, Lanny Smoot. Smoot, Imagineer Gary Schnuckle and Timothy Caldwell are the driving forces behind a new patent application which seeks out the next generation of reliable, consistent, automated performances and they just might have found the answer for 2010 in technology whose origins have been traced as far back as 2000 B.C. — marionettes.
Now we aren’t talking about marionettes in the traditional sense, because that just wouldn’t be patent-worthy or (let’s be honest) time-worthy. We are talking about bringing the traditional art form to a much larger scale. Life-size puppets attached to several strings, manipulating their every move on a full-size stage in front of a live audience. Up until now, the closest performances have come to being able to provide this form of entertainment is the Japanese art of Bunkaru in which the puppeteers wear all black to blend in with the background so that their large puppets appear to move on their own without any sort of human intervention – almost.
The patent application, titled ‘Robotic Marionettes on Magnetically-Supported and Highly Mobile Puppeteer Platforms,’ describes a system in which the life-size marionettes are attached to a device called the puppeteer vehicle. The puppeteer vehicle, in turn, is magnetically attached to the tender vehicle, the two of which being separated by a thin membrane of sorts, which essentially amounts to a physical ceiling to the set. The tender vehicle on top of the membrane/ceiling is programmed to move in specific positions (most likely controlled wirelessly) and drags the puppeteer vehicle with it via the magnetic connection. The puppeteer vehicle, in turn, contains all the mechanical elements to be able to manipulate the large puppets. In certain cases, such as the case of eye movement, robots can be installed on the puppet itself, providing some of the benefits of animatronics/robotics to the anti-technological puppet.
Applying the technology to the traditional art of marionette puppetry has an immediate two-fold benefit. Firstly, while the puppets might be able to walk (or fly) across a large stage with humans in control, they are limited in the Z axis. That is, they could never negotiate the difference between up stage and down stage. Because of the thin membrane which requires no strings to pass through it, the magnetic system would allow the puppets to walk towards and away from the audience. Second, and more importantly, the system allows for the puppets to approach and interact with each other without risking the puppets colliding and/or strings becoming entangled. Although robotic systems have been developed to manipulate the puppets, it’s impossible to have multiple puppets move around each other because of the physical nature of the robotic arms themselves.
Sorry, Pinocchio, but you’ve had your fun.
The 1313 Club is hosting an event with Ape Pen Publishing on September 18 at the Disneyland Hotel in Anaheim, California. Walt Disney’s Marvelous Mechanized Magic Kingdom will bring together Disney Legends such as Bob Gurr, Alice Davis, Blaine Gibson, Rolly Crump, X Atencio and more alongside actual audio animatronics from Garner Holt Productions.
The event will illustrate Disney’s incredible mechanical achievements by spotlighting attractions, characters, and memorable devices created by the Disney Studios and, later, the MAPO arm of WED Imagineering. Using rare images, video, and authentic props and animatronics, guests will gain insight into the fascinating MAPO division as never before. Disney Legends will shed light on this little known but highly intriguing aspect of Disney history—plus performances by Disneyland entertainers and the World premiere of the world’s most advanced animatronic figure.
This once-in-a-lifetime evening will honor Disneyland as one of the world’s great storytelling environments, with special emphasis on its technological wonders. Combining history, entertainment, and lots of surprises, it’s an event not to be missed by any Disneyland Fan.
Prices for the event range from $82 to $120. For additional information or to register for the event, see here.
ANAHEIM, Calif. (Dec. 18, 2009) — President Abraham Lincoln returns to the Main Street Opera House in Disneyland with stunning new Audio-Animatronics technology that makes this the most lifelike and expressive Lincoln figure yet.
Great Moments with Mr. Lincoln, a beloved Disneyland attraction for nearly 45 years and a historic landmark in the development of Audio-Animatronics technology, reopens Dec. 18 on Main Street, U.S.A. with the new Lincoln figure and an enriched presentation that combines the best elements of the park’s original 1965 show and the most advanced technical enhancements. (The show first opened at the 1964 New York World’s Fair.)
The new Abraham Lincoln figure represents several major milestones for Audio-Animatronics figures. The fully electric head – a first for human Disney Audio-Animatronics figures, which are traditionally hydraulically operated – is based on an all-new sculpt that is more lifelike than in previous versions of the show. With an expanded range of facial movement, Lincoln is more expressive than ever. He can purse his lips, form an “O” with his lips, smile, grimace and use his eyebrows to enhance his emotions.
For five days at Epcot in 1999 and for an equally short time in Castaway Cay, guests from around the world had the chance to meet with one of the largest accomplishments in audio animatronic technology: the DRU-1 (Dolphin Robotics Unit). Created by Edge Innovations in partnership with Walt Disney Imagineering, DRU-1 wowed the crowds but was ultimately decided to not have the potential of being an every-day attraction in the theme parks.
Here’s a video of DRU-1 in action courtesy of his WDI show producer:
And for more a lot more great information on the Walt Disney World project told from the perspective of someone who was directly involved, check out DisneyShawn’s blog here.