Multi-log grasping using reinforcement learning and virtual visual servoing
Wallin, E., Wiberg, V., and Servin, M. Multi-log grasping using reinforcement learning and virtual visual servoing Robotics 13(1):3, (2024). [doi:10.3390/robotics13010003] [pdf]
We explore multi-log grasping using reinforcement learning and virtual visual servoing for automated forwarding. Automation of forest processes is a major challenge, and many techniques regarding robot control pose different challenges due to the unstructured and harsh outdoor environment. Grasping multiple logs involves problems of dynamics and path planning, where the interaction between the grapple, logs, terrain, and obstacles requires visual information. To address these challenges, we separate image segmentation from crane control and utilize a virtual camera to provide an image stream from 3D reconstructed data. We use Cartesian control to simplify domain transfer. Since log piles are static, visual servoing using a 3D reconstruction of the pile and its surroundings is equivalent to using real camera data until the point of grasping. This relaxes the limit on computational resources and time for the challenge of image segmentation, and allows for collecting data in situations where the log piles are not occluded. The disadvantage is the lack of information during grasping. We demonstrate that this problem is manageable and present an agent that is 95\% successful in picking one or several logs from challenging piles of 2--5 logs.
Videos

This work was supported by Mistra Digital Forest, Algoryx Simulation AB, Cranab AB, and eXtractor AB. The simulations were performed on resources provided by the Swedish National Infrastructure for Computing at High Performance Computing Center North (HPC2N).

UMIT Research Lab, Digital Physics