What is Motion Capture?

Motion capture or mo-cap for short, is the name for technology that records the movement of people or objects. The motion is captured either by hardware such as optical cameras or inertial sensors, or can be processed using video input into an Artificial Intelligence (AI) system. The data captured is then transferred to a computer program to be used in a virtual environment.  Mo-cap has become increasingly popular in recent years, as a method of combining computer-generated 3D characters with the natural and often subtle movements or mannerisms of human actors.  Motion capture sessions involve capturing the movements of actors. Only the movements are recorded - not their real visual appearance. The animation data is then mapped to a 3D model which is enabled to perform the actions that were captured.  

How Does Motion Capture Work?

Optical Tracking using Cameras

With optical tracking, special infra-red cameras are used to visually track the movement in a given space, known as a tracking volume. They look for either retroreflective markers to bounce the infra-red light back into the cameras, or LED markers which emit a pulse frequency of light directly into the cameras.

Multiple cameras are required to accurately track markers in a volume, as at least 2 cameras are needed to sufficiently calculate the 3D position of a marker in the real-world space. They can track this position down to sub-millimetre accuracy. Additional cameras allow for more accurate data to be captured, as more cameras that can see 1 marker helps with the technology to calculate its 3D positional data. Different perspectives also help with accuracy in the event of an object blocking the camera’s view of the marker, which is referred to as occlusion.

Having more cameras helps with accuracy but depending on the scenario there will be a limit to this improvement and can be expensive for not much additional benefit. Typically, additional cameras would be spaced apart to allow tracking of larger volumes, while also overlapping to support with increased accuracy of tracking, to maximise efficiency and value.  

Typically to capture the actions of an actor, they would wear a special suit made from a soft, form fitting Velcro material. The markers would be placed on the actor in positions that correlate to their joint and bone locations, in a pattern that matches with the motion capture software. This then creates a digital skeleton that would then be used to control a 3D model.

Optical tracking can be divided into 2 categories:

Optical Passive Motion Capture

Optical passive is the most flexible and commonly used method of motion capture. It uses the infrared cameras to track retroreflective passive markers by bouncing the infrared light back into the cameras. It allows for effective and accurate tracking of data, while the cost of the markers low. The markers are flexible to allow for a variety of actions to be captured. This can have issues where other retroreflective items in the volume, such as shiny metallic objects like a car or staging, or glass like objects like windows can confuse the camera by generating reflections back to the camera to create false markers. It can also affect the quality of tracking of the retroreflective markers, and best practices are to remove any items that are not required for the shoot. If items are required, try to remove any reflections they might generate.  

Optical Active Motion Capture

Optical active uses cameras track LED motion capture markers as they emit light. This is often associated with a higher cost, as the markers must be able to emit their own LED light in a frequency to match the camera to track. This is more accurate to track the markers as the marker is emitting its own beacon so not relying on the light to reflect into the camera. It also has greater accuracy as the infrared LEDs can be switched off from the cameras to greatly reduce any issues from reflective objects in the volume.  

As the object needs to have its own power source and to be able to match the LED frequency of the motion capture system, this adds costs to the marker. They are also larger due to the additional components or batteries required to operate, and more rigid than the retroreflective markers. Retroreflective markers are much more durable and hardwearing, which is why they are preferred to be used for actor movement tracking. Active tracking LEDs are preferred for objects, especially fast-moving drones as it allows more accurate tracking.

Inertial Suits

Another form of motion capture is inertial suits, Where the actor wears inertial sensors (also known as IMUs) at specific joints and parts of the body. These sensors detect the motion and transmit data wirelessly to a computer or smart device. The software calculates the end motion using the combined data it receives from the sensors. Inertial suits are often a more affordable option for motion capture than optical tracking, as the costs for the sensors are much lower than the optical cameras.  

As the sensors are transmitting only their individual movement data, it can sometimes be more likely to create errors or inconsistencies with the data, and can be harder to clean up after. Magnetic interference can also impact the data, as well as the distance the sensors are from the computer or smart device so these are factors to consider, but overall, this form of mocap can have greater flexibility as you need less equipment to capture the motion.

Virtual Reality

Using Virtual Reality for motion capture is also possible. This tends to work by using the headset’s rotation and physical location as the position for the head of the mocap animation. It then uses the controllers or VR trackers (such as Vive Pucks) to help with other main body positions such as hands or feet. The software then takes your height measurement, and interpolates the information for all the body parts and joints in-between your head and hands. It uses an average body as a template, so it might not track as exact, but is a low cost option for creating motion capture data. Some software allows you to change some of this interpolation information to give you more realistic data.  

Video / Markerless Mocap

Mocap without suits or Video/markerless systems are also available as an alternative form of motion capture. Instead of markers, this technique relies on software and video input. This technique does not require markers and instead relies on software to track the subjects’ movement. Varying tracking methods yield different results, but real-time and final data error ranges tend to be larger than marker-based solutions.

These systems rely on AI to analyse video footage and generates mocap data from that (with support from a dataset). Issues with video clarity, objects that block the person and length of video can impact the success of the motion capture data being generated.  Some systems only allow 1 person in a video to process mocap data, which inertial and optical systems can allow for many people to be tracked at the same time. Clothing can cause issues with this form of motion capture too, as if it covers joints or the clothing itself it loose then can confuse the mocap data being captured.

Video/markerless are a systems great entry point in the world of motion capture, as well as a handy tool for pre-visualisation.

Even though the ease of use, price and data quality make markerless systems very appealing, there will still be many situations where robust mocap tools such as optical or inertial will be needed. These include:  

• Data quality: This is especially important for more complex motions, inertial or optical mocap provide higher fidelity capture.
• Occlusion: The A.I. relies on a complete view of the performer to estimate the skeleton's position. This means that occluded limbs will translate in poorer tracking capabilities, either because the performer is outside of the video frame or because the position of the body of the performer makes it more difficult for the A.I. to make an estimate. With marker based mocap, this is not an issue at all.  
• Real-time vs post-processed data: With A.I. motion capture, post processing is heavy but needed to generate the animation file, meaning it's very hard (unless you cut some big corners) to generate the animation in real-time.  
• Face and finger capture: This require more precise tracking which inertial or optical can offer. Some AI systems can process facial video to create mocap data, but results can vary.  
• Tracking space: Background and lighting are important to capture a clear video (and thus a good animation), as is distance to the camera. Inertial and optical tracking can offer more flexibility in this regard as well (and lighting or other environmental factors are irrelevant).
• Multiple performer recordings: Currently AI and markerless systems find it difficult to process the animation data of multiple people in one video at the same time. Inertial and Optical allow for multiple actors to be in the same space captured at the same time. This allows for more realistic animation interactions between different actors.  

Advantages vs Disadvantages

Motion capture is a brilliant way of creating realistic and interesting characters off the back of real-life actors, but it doesn't come without some challenges. So, what are the advantages and disadvantages of using motion capture?

Advantages of Motion Capture

• Low latency (almost real time) results, reducing the cost of keyframe-based animation.
• The amount of work it takes isn’t dependant on complexity or performance length as traditional techniques. Different styles and deliveries can be tested, with personality only limited by the actor’s talent.
• Realistic, complex interactions and movements (e.g. weight and exchange of force) can be easily recreated in an accurate way. 
• Large amounts of animation data can be produced when compared to traditional animation, helping with meeting deadlines and cost effectiveness.

Disadvantages of Motion Capture

• If making a small production, cost of equipment and software can be prohibitive.
• Specific software and hardware needed to obtain and process the data.
• Camera field of view (obstructions)  or magnetic interference (inertial suits) may affect the capture system’s specific requirements

Final Thoughts

If you would like to learn more about motion capture and the facilities that we have at PROTO in Gateshead, please get in touch with our Studio Manager, Sophie Craggs at: sophie.craggs@sunderlandsoftwarecity.com

‍