Digital Twin Builds Better Robots

Robots have risen to unprecedented prominence, as manufacturers and warehousers turn to robotics to meet market pressures that have overwhelmed traditional technologies and practices. These robots, however, are not the machines of the past—physically intimidating systems, with limited intelligence and mobility that do not play well with humans. Current robots are nimble systems that support greatly expanded intelligence and perceptive capabilities, and that can perform complex and sophisticated tasks while they work alongside humans.

To build this new generation of robot, engineers need technologies and practices that provide greater visibility, flexibility and efficiency. To contend with previously unheard of levels of complexity, all engineering disciplines must work together in unprecedented collaboration, leveraging data-rich environments.

This is where the digital twin comes in. Armed with the enhanced modeling and simulation that digital twin technology brings, engineers are better equipped to design, develop, deploy and monitor robots in manufacturing and warehouse applications (Fig. 1).

“We now have the technology to create true-to-reality digital twins of the physical world,” says Erin Rapacki, senior product marketing manager for robotics at NVIDIA. “Simulation in digital twins of manufacturing and warehouse logistics facilities will allow operators to optimize efficiency and significantly reduce cost and waste.

To conduct these complex simulations, Rapacki says, it’s necessary for users to simulate all digital twin aspects, including the robots, sensors, people and other assets such as conveyors.

“The robotics developer will be able to exercise the complete software stack—including the safety systems—thousands of times in simulation before ever deploying to the real world,” he adds.

Achieving Painless Design

When developing new industrial robotic systems, development teams want to quickly try out different designs and control ideas and lock down final product requirements. With access to a digital model of the physical system, there is a virtual testbed to run through design iterations, with lower cost and lead time than when creating physical prototypes each time (Fig. 2). This allows design engineers to cast their nets further, committing time and resources to exploring concepts and approaches that traditional means could not afford.

“When creating new innovative uses, this broader analysis allows corner cases to be considered that would normally be discounted as taking too long to evaluate,” says Chris Harduwar, vice president of strategic solutions at Maplesoft. “As the digital model is kept up to date with operational data, a fully rounded digital twin is formed that responds like the physical asset. The digital twin model can also be used as a flexible platform for verifying control strategies and updates planned for the production line, all resulting in a safer, more robust product.”

To experience the full rewards promised by a digital twin, however, development teams must consider the full picture, including all disciplines involved. Digital twin models and simulations can also help.

“A digital twin of a manufacturing or warehouse logistics facility enables digital connections between all activities in the workspace and communication of changes like traffic, obstacles, optimal routes and updated pickup/drop-off points to the robot installment,” says Rapacki.

For example, when a mobile robot uses the same simulation environment as the facility’s digital twin, the mobile robot more easily integrates with the real-world environment. The robot’s sensor stack leverages the actual digital twin of the customer site for robot perception training and testing. The customer site also provides synthetic data for computer vision training, and on-robot vSLAM (visual simultaneous localization and mapping) helps the robot navigate according to updated facility models.

Leveraging Offline Programming

The complexity of robotics programming—as well as the high cost of debugging a robotics system on the warehouse or production floor—make digital twin-enabled modeling and simulation essential ingredients in sophisticated robot application design, deployment and maintenance. Once the robot and the associated processes are designed and validated, engineers can use the modeling and simulation tools to perform offline programming.

Digital twin-enabled modeling and simulation allow design teams to develop and test the code before the complete physical system exists (Fig. 3). This saves time by allowing programming of the code and construction of the physical system to be done in parallel.

“The outcome ensures optimized cycle times and a collision-free trajectory,” says Roberta Tanzariello, platform lead, robotics and digital tools at Rockwell Automation. “Advanced robotics simulation software enables the deployment of challenging robotics applications like welding, drilling and cutting through offline robotics programming. Users can add detailed information to create complete programs offline.”

Once an initial digital model is created and tuned using first-pass nominal data, it is ready to be integrated with the control system development tools.

“For manufacturing automation, control engineers can prepare a virtual [programmable logic controller (PLC)] and use the signals as inputs for the digital twin, which becomes an environment where the engineer can verify and simulate the physical response to the new code,” says Harduwar.

Offline verification is valuable for preliminary integration testing on systems that will operate under hazardous work environments or where a code error could cause harm to the robotic system or operator. In this way, testing fault conditions is safer and can be done more rapidly in virtual environments.

On existing robotic systems, engineers can investigate problems via online simulation, and use the digital twin to represent the production environment. This works in the same way that a digital twin can evaluate how code updates perform.

“We’ve seen this used for troubleshooting anomalies reported by customers who have seen the problems in hardware running on their sites,” says Harduwar. “Once the root cause is identified, the model can be used to validate the expected outcome before applying the programming changes to a live system. This greatly reduces the system downtime and safeguards against an expensive impact of errors in the code.”

Expanding AI Training Possibilities

Digital twin technology can also help with training AI algorithms, mechanisms that now enable robots to perform sophisticated applications in manufacturing and warehouse environments.

Until recently, algorithm training has often been hamstrung by the scarcity of training data. Often, robotics development teams have found themselves in situations where training data was nonexistent, significantly limiting the applications that algorithms could be trained to perform.

The challenge of training algorithms has been further complicated by the sheer number of unique applications robots are asked to perform, which has exasperated the training data problem.

Digital twin technology, however, now offers a way to train AI algorithms to take on greater numbers and types of use cases. A digital twin can produce simulated data and test new functionalities, quickly going through numerous configurations and designs. The data produced by these simulations can then be used to train AI models, accommodating a broader variety of real-world conditions and testing scenarios.

“Using synthetic data generated by digital twin simulations, AI algorithms can learn and adapt to different conditions and variations in the physical world,” says Alex Greenberg, director advanced robotics simulation, Siemens Digital Industries Software. “This approach helps to reduce the risk of potential issues before they occur in the real world.”

An example of this can be found in the recent domain of 3D computer vision (that is, machine perception and sensor fusion), which is essential for robot manipulation and understanding object distances.

“Object distance is critical to robots’ ability to manipulate, grasp, or avoid an object,” says NVIDIA’s Rapacki. “A robot must understand how far away the object is. Digital twins provide endless scenarios with geometric features that a 3D robot perception pipeline can use to train for collision avoidance and grasping.”

Pushing the limits of AI technology further, a technique called “reinforcement learning” uses digital twin technology to solve even the most complex problems. The technique enables an algorithm to learn how to perform a task through trial and error in a simulation, a process that would not be practical in the real world.

Overcoming Commissioning Roadblocks

Another area where the power of digital twin modeling and simulation can be brought to bear is in robotic system commissioning and deployment.

Though these processes are hindered by obstacles, the major bottleneck with which engineers must contend is the need for more access to physical systems by those who must verify the robot’s functionality.

Unfortunately, it is costly and often impractical to build and operate a physical system, so another option is required. This is where digital twin technology comes into play by enabling virtual commissioning.

This process—using a representative customer environment in simulation as a digital twin—addresses these challenges and improves efficiency.

Virtual commissioning allows engineers to connect the digital twin to a control system (for example, a PLC) and other external devices to test, refine and optimize mechanical, electrical and logical designs before any hardware is available and without the need to stop production.

Hardware-in-the-loop simulation can be performed as part of the virtual commissioning process. Here, different device types and systems can be combined in a complex system layout to provide the development team with a holistic view of the robot’s performance in the operating environment.

The PLC interfaces with the robot, and the complete deployment is considered in a detailed simulation, where gravity, force and torque are factored in to size the application according to the cycle time required. In this case, the simulation tests real control input from the PLC and feedback from the simulation model.

Through the virtual commissioning process, engineers are often able to identify and correct programming bugs and mechanical interference. The process also verifies the dynamic zone and the robot’s work envelope to ensure safe operation.

“Using simulation software in this way, robotics engineers can rapidly produce different virtual prototypes much faster than building a single physical system,” says Maplesoft’s Harduwar. “It is also easier to test the performance of a digital model under a variety of operating conditions than to set up multiple testing spaces in a warehouse.”

Digital twin-enabled virtual commissioning helps robotics development teams achieve a high level of accuracy, and it provides the means to validate not only individual devices and subsystems, but also how well all of the constituent parts work together.

“Simulations allow robot engineering teams to come as close to perfection as possible in validating corner cases and performing bug fixes, merging code and managing behavior trees and deployment integration,” says Rapacki. “This systems-level validation—particularly for code updates—accelerates the deployment time and significantly improves the quality of robots.”

When Digital Twin Goes into the Field

Once the robotics development team has prepared the digital twin, it can extend the simulation to applications beyond the design and initial commissioning stages (Fig. 4). If real-time operational data is available from sensors or can be extrapolated from historical data, the development team can augment the digital twin so that it represents the physical robotic system over time. This can support performance optimization to drive business outcomes by tweaking the controllable parameters to maximize throughput and find ways to reduce operational wear.

“Robot performance can be evaluated and tested based on real behavior observed during operation, and digital twin simulation can be used to look forward to support predictive and prescriptive measures to improve performance or evaluate alternative solutions to potential issues,” says Mike Rouman, senior marketing manager, digital manufacturing at Siemens Digital Industries Software.

For robot assets that need to respond to a changing environment or to interact with other machines, the digital twin can serve as an ideal operating case, from which designers can measure anomalies and consider production changes. This process simplifies product improvements and processes and the ability to adapt when performance parameters start to fluctuate.

The industry is now witnessing digital twins being used to troubleshoot operational hardware, as the model provides context for problems. This enables designers to work on any remedy and code changes without bringing the affected system offline. With the right quality of sensor data and by speeding up simulation runtime, the digital twin can study the system from more vantage points than standard fixed-time simulations.

“Future challenges will be to develop fast, efficient digital twin models across a broader range of environments, and to train more engineers to learn to use and apply the tools that support these digital transformation techniques,” says Harduwar.

More Maplesoft Coverage

More NVIDIA Coverage

More Siemens Digital Industries Software Coverage