On the basis of a few photos alone, a new technique creates realistic avatars of animals that look and move like real animals. This could be helpful to assess endangered animals in the wild as part of species protection projects.
Filmmakers and developers of computer games will have a new way of animating animals in the future. A team led by researchers at the Max Planck Institute (MPI) for Intelligent Systems in Tübingen has developed a technique that uses photographs alone to create lifelike 3D models of almost all quadrupeds. These avatars can be animated to realistically imitate the movements of animals. But the simple method of bringing animals to life on the computer is not only interesting for the entertainment industry. It could also benefit biologists in species protection and help to make children in particular aware of the importance of biodiversity. The new technique and its results are published as part of the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 20 June 2018. And the model is made available for non-commercial and scientific research purposes (see SMALR).
The Tübingen researchers’ technique uses photos of animals (bottom row) to create 3D models (top row), which can reproduce different postures and whose surface is designed according to the original body cover (middle two rows).
© MPI for Intelligent Systems
Michael J. Black, co-author, director Perceiving Systems department, Max Planck Insitute for Intelligent Systems, Tübingen, Germany.
“Populating the virtual world with animals used to be a lot of manual work“, says Black, director of the Perceiving Systems department at the MPI for Intelligent Systems. Today, animators have to laboriously design 3D animals using graphics software. While there are technologies to create realistic 3D avatars of humans, no such technology is available to automatically capture the shape of wild animals. “You can’t bring a tiger into the lab to be scanned and you can’t take expensive scanners out into nature,” says Black.
The key is to start with a rough 3D shape
The team, whose technique creates avatars of animals including their fur or skin with reasonable effort, includes scientists from the CNR Institute for Applied Mathematics and Information Technology (IMATI) in Milan and the University of California in Berkeley as well as Max Planck researchers. “Ultimately, 3D animal capture is about reconstructing the 3D structure from 2D images of moving objects captured by various unknown and moving cameras,” explains Silvia Zuffi, the lead author on the project from IMATI.
In their new method, the researchers only need a few photos of an animal in different views to calculate realistic models that can then be animated naturally. “You could make a fortune doing this for people’s pets”,” says Michael J. Black.
Unlike other approaches, the images from which the avatars are generated can also come from different cameras. The computer programme delivers such convincing results because it doesn’t start from scratch when it creates a model. “The key is to start with a rough 3D shape model that can explain the shapes of many animals and then refine this based on the observed images,” says Michael Black.
Helping endangered animals in the wild
Tasmanian tigerIn their current work the researchers demonstrate that their method works for as different animals as a tiger, a bear, a horse or a rhino. “Our algorithm generates realistic models of animals for which no previous 3D models exist,” says Silvia Zuffi. For example, the researchers were able to reconstruct the 3D shape of the extinct Tasmanian tiger from low-quality film footage shot in the 1930’s.
(Source: Max Planck Gesellschaft news release, 21.06.2018)
