Semi-Autonomous Crane

Industrial crane manipulators are widely used in mining, marine and forest environments. These are used for heavy and dangerous operations associated with the extraction, transport, and processing of raw materials and for various rescue operations. The electro-hydraulic and mechanical properties of large industrial manipulators make them difficult to control for both operators and using conventional control methods developed for industrial robots. Fast and stable motion control and semi-autonomous functions has emerged only recently in commercial systems. The lack of robust computer vision, however, severely limits the development of higher level autonomy, for example, ability to grip and move objects and to plan and predict the outcome of movements in unstructured environments.

We explore the use of computational modeling and multidomain dynamics with frictional contacts in the development of fully and semi-autonomous cranes. The research also address geometric design optimization of crane manipulators based on the computational model and simulation output.

Rigid multibody dynamics model

The crane system is modeled as a multibody system using non-smooth dynamics. The crane model consists of 13 rigid bodies (2 for the base, 3 for the booms with 2 bodies as connections in-between and 3 hydraulic cylinders of 2 bodies each), and an additional 8 for the rotator and grapple, connected using kinematic constraints (hinge and prismatic) and actuated using secondary constraints (hinge motors and prismatic linear motors). These joints are made non-ideal by assigning compliance, viscous damping, and joint end-limits. The constraints motors are given force limits not to make the manipulator unnaturally strong. The actuation is controlled by assigned a desired joint velocity at each time-step. The resulting joint velocity depends on the inertia, dynamics, joint compliance and actuator force limits.

Logs of different size and length were created for the crane to lift. The logs are modeled as cylindrical rigid bodies with realistic mass density, friction, and restitution. The contact model is based on non-penetration, Coulomb friction and Newtons law of impact.

Motion planning and tracking for automatic loading

The system is assumed to be equipped with ideal vision sensors that give the precise location of the logs and the tray. A motion planner algorithm generates smooth target trajectories for the gripper in Cartesian space, e.g., to place it above the center of a log or a specific point above the load tray. The Cartesian velocity is translated into joint velocities using inverse kinematics. Joint limits are respected by the motion planner by including weight factors in the Moore-Penrose pseudo inverse as function of joint coordinates

A motion tracking algorithm compares the current position and velocity with the planned trajectory and generates a new motion plan from this. The Cartesian motion plan is represented either as linear segments or by cubic splines.

Co-simulation

The full system is simulated in co-simulation manner, with the multibody dynamics using AGX Dynamics and the motion planning and path tracking using MATLAB. AGX outputs the position and velocities of selected bodies (ideal sensors) and joints. MATLAB outputs target joint velocities. Example video showing interactive Matlab for controlling the crane.

Machine vision and perception

The lack of robust computer vision, however, severely limits the development of higher level autonomy, for example, ability to grip and move objects and to plan and predict the outcome of movements in unstructured environments. We explore model-based vision of objects in the manipulator's operating environment, such as tree logs, chains, pipes or rocks and boulders. The idea is to combine camera techniques for computer vision with realtime physics simulation. By fusing the data streams from cameras and other sensors with data from multiple parallel simulations the physically impossible interpretations can be excluded and object tracking become robust. When the input is insufficient, the crane manipulator can perturb the environment to generate a dynamic response that reveals additional information about the objects' geometry and physical nature.

Semi-autonomous control

Semi-autonomous operation with shared control between the human operator and control computer has been developed and examined with human operators and a lab crane at the Robotics and Control Lab at Umeå University. The results are published here

Topology optimization

The crane dynamics is computed using physics-based joint compliance and no coordinate reduction is employed. This means that all forces and moments that occur during a loading cycle can be registered and analyzed. This can be repeated for a large set of loading cycles. Repeating the loading motion for multiple cycles, statistics of the forces and moments on the different parts can be produced. We investigate this workflow and the use of dynamic forces can be used for further analysis, e.g. as input forces for structural analysis or design optimization.

Publications and material

• Servin M. Datorseende i granulära processer med realtidsfysik. Slutrapport från genomförbarhetsstudie inom det strategiska innovationsprogrammet Processindistriell IT och Automation (2017).

• Servin M. Simulatorexperiment - datorseende skutknackare genom realtidsfysik. Umeå universitet (2017).

• Hansson A. and Servin M. Semi-autonomous shared control of large-scale manipulator arms. Control Engineering Practice, Elsevier, Vol 18, Issue 9, pp. 1069-1076 (2010). pdf, web

• Hansson A. and Servin M. Semi-autonomous Shared Control for Redundant Forwarder Cranes. Project report, Umeå University (2008).

• Hansson A. Shared Control of Mechanical Systems in Virtual Environments. MSc Thesis, Umeå University (2009).

• Simulation scripts: grippingBoxes, craneSimulation, monitorControl