Replicating Dynamic Humerus Motion Using an Industrial Robot

Summary

Unlike a traditional socket prosthesis, an osseointegrated (OI) prosthesis attaches directly to the bone of the residual limb. OI prostheses provide upper-extremity amputees increased range of motion, more natural movement patterns, and enhanced proprioception. However, the direct skeletal attachment of the prosthesis elevates the risk of bone fracture. To minimize the risk of fracture, it's important to mechanically characterize the bone-prosthesis interface under the same conditions that it would experience in vivo. In this project, I robotically replicate the motion of the humerus as recorded via motion capture while subjects performed activities typical of an active amputee. The robotically replicated motions will be utilized in future investigations to mechanically characterize the bone-prosthesis interface of an OI prosthesis.

Code and Demos

Below you will find a short video of the robot utilized in this investigation performing 3 jumping jacks. The white bone in the video is a 3D printed humerus. The hemisphere is comprised of a collection of LEDs that can be optically tracked using an Optotrak measurement system. By optically tracking the hemisphere, I can measure how accurately the robot is able to replicate the desired humerus trajectory (recorded via motion capture from a human subject performing jumping jacks). The beginning of the video demonstrates an automation utility that I created, which allows the user to select a particular motion capture trial for robotic replication. Once the trial is selected, the utility turns on the Optotrak measurement system, instructs the robot to run the appropriate trajectory, records the trajectory of the humerus from the hemisphere LEDs, and writes all collected data to disk.

The code for this project is split amongst 4 repositories. To make it easy to access and reference the code in the associated manuscript, I linked the GitHub repositories below to Zenodo. This setup provides me a Digital Object Identifier (DOI) for each repository and more specifically for each release.

Motion Capture to Robot software package	MATLAB, Python
The code in this repository serves as a thin wrapper for reading input files, executing algorithmic logic in the supporting library (see below), and outputting files. The README of this repository provides an overview of how the 4 code repositories are related and the rationale behind this organization scheme. The code is largely written in MATLAB, but some plotting functionality is written in Python.

Motion Capture to Robot software library	MATLAB, Python
The code in this repository provides a set of algorithms for processing motion capture trajectories (augmenting, smoothing, subsampling, etc.) and for writing motion programs to the industrial robot utilized in this project FANUC M20iA. The code is largely written in MATLAB, but some plotting functionality is written in Python.

Derivative Free Trajectory Optimization	C++
Robot Operating System (ROS) package that utilizes the Constrained Optimization BY Linear Approximation (COBYLA) algorithm to optimally map a motion capture trajectory from the motion capture lab reference frame to a robot joint angle trajectory. The README of the repository provides more details on this package. The manuscript, which I will post on this page once accepted for publication, provides a detailed mathematical description of this algorithm.

Gradient Based Trajectory Optimization	C++
ROS package that utilizes the Sparse Nonlinear OPTimizer (SNOPT) to optimally map a motion capture trajectory from the motion capture lab reference frame to a robot joint angle trajectory. The README of the repository provides more details on this package. The manuscript, which I will post on this page once accepted for publication, provides a detailed mathematical description of this algorithm.

Current Status

• All work has been completed and manuscript was published at PLOS One.

Background

Utilizing robots for biomechanical testing is not a novel idea. In fact, the field is sufficiently advanced that commercial software/hardware solutions (e.g. simVitro) can now be utilized for biomechanical robotic testing. The approach utilized in this project is fundamentally different than that of other studies because it replicates the motion (kinematics) of the humerus, which in turn will impart forces and moments at the bone/prosthesis interface. From this point forward, I will refer to the combined forces and moments as loads. The alternative (conventional) approach is to numerically compute the loads at the bone/prosthesis interface using inverse dynamics, and program the robot to replicate the calculated loads (termed force control). The advantage of the force control approach is that specified loads (and their directions) can be accurately replicated; the disadvantage is that replication of the rate at which loads are applied to the bone/prosthesis interface will be extremely poor. However, the magnitude of the load that is necessary to fracture a bone is determined both by the direction of the load and how quickly the load is applied (load rate).

Replicating kinematics (the approach of this study) has the potential to match the load replication accuracy of the force control method while also accurately replicating the rate of load application. In practice, however, the method by which industrial robot are programmed presents roadblocks to realizing this potential. When programming an industrial robot one has to balance position accuracy (analogous to load direction) and velocity accuracy (analogous to load rate). Practically, this limitation dictates that - by utilizing the approach of this study - it is necessary to balance the accuracy of load direction, magnitude, and application rate. Our results indicate that, on average, it is possible to simultaneously replicate load direction to within 95% accuracy and load magnitude to within 90% accuracy. Load rate application accuracy is extremely difficult to measure using current technology; but, qualitatively, replicating kinematics provides at least 10 times better accuracy than the force control method.

The necessity to balance between position and velocity accuracy is not dictated by physical robot limitations, but one imposed by the programming language employed by industrial robots. Industrial robots are typically employed in pick-and-place or assembly operations, and as such their programming language is designed to facilitate these scenarios. Industrial robots are very rarely utilized outside of factories; however, a movement to utilize robots outside of the typical factory settings exists. The ROS-Industrial Consortium aims to facilitate the use of industrial robots outside of the factory setting, and in particular, to enable the utilization of industrial robots in research settings. If an advanced programming interface is provided by robot manufacturers in the future, then the methodology proposed in this study can replicate load magnitude, direction and rate as measured in vivo.

The optimization algorithm that maps the humerus trajectory from the motion capture lab reference frame to a robot joint angle trajectory is the key ingredient that enables the robot to replicate humeral kinematics. Although two optimization algorithms are listed, they differ only in their mathematical formulations of the optimization problem - their objective is identical. The manuscript describes these algorithms in mathematical detail, so here I want to provide only a general understanding of their utility and essence.

The end goal of robotically replicating human motion is to reproduce the loads that the bone/prosthesis interface experiences in vivo. The loads at the bone/prosthesis interface are caused by the velocity (centripetal torques) and acceleration of the humerus, and gravitational forces. This means that when robotically replicating a trajectory we not only need to reproduce the velocity and acceleration of the humerus, but also its orientation with respect to gravity. However, if a trajectory is modified in specific ways, the humerus will still maintain its original orientation with respect to gravity. A simple example will suffice to illustrate this. Stand up and perform a jumping jack. Now, imagine that you had performed the same jumping jack a few feet forward, backwards, to your left, or right. The orientation of the humerus with respect to gravity would have been the same. You could have also gone one floor up or down, or faced a different wall in your room while performing the jumping jack, without affecting the orientation of the humerus with respect to gravity. Succinctly, we could say that the orientation of the humerus with respect to gravity is invariant to all translations and rotations around the gravitational axis. Because the robot is fastened to the floor (and due to its kinematic structure), changes in the trajectory dictate both the effort the robot must expend in performing the trajectory, and whether it can successfully complete the trajectory or not. The optimization algorithm translates and rotates the original trajectory around the gravitational axis to find a new trajectory that the robot can execute with minimal effort.

This was my first major Ph.D. project, and it was a fantastic learning experience. I gained much experience in robotics, linear algebra, rotations, optimization algorithms, kinematics, dynamics and signal processing. I also learned how to use SolidWorks to design the fixtures for attaching the humerus to the robot, and to design the hemisphere for housing the LEDs. On the software side, I re-learned C++, having first learned it in my computer science class in high school. I became much more proficient in Matlab and starting using Python for plotting. Apart from these tangible skills, however, I learned how to shepherd a somewhat complex scientific investigation/project from beginning to end.