Recently, the research Center for Biodiversity Intelligent Monitoring of China State Key Laboratory of Efficient Production of Forest Resources led by Professor Zhang Junguo from the College of Technology has made new progress in autonomous robot movement decision-making. The research results entitled "Event-triggered Reconfigurable Reinforcement Learning Motion-planning Approach for Mobile Robot in Unknown Dynamic Environments" was published in Engineering Applications of Artificial Intelligence, a well-known international journal in the field of artificial intelligence (Q2, IF=7.802).

In recent years, the combination of robotics and reinforcement learning has become one of the cutting-edge technologies in the field, which has gradually promoted the transformation of human social production and life style. Reinforcement learning uses data-driven methods to improve the ability of robots to cope with complex environments, but it  is accompanied by high learning costs and instability of autonomous decision-making.

Deep reinforcement learning (DRL) is an essential technique for autonomous motion planning of mobile robots in dynamic and uncertain environments. In attempting to acquire a satisfactory DRL-based motion planning strategy, the mobile robots encountered several difficulties, including poor convergence, insufficient sample information, and low learning efficiency. These problems not only consume plenty of training time, but also bring a negative impact on motion planning performance. One promising research direction is to provide a more effective network framework for DRL-based policies. Along this line of thinking, the paper presents a novel DRL-based motion planning approach called Reconfigurable Structure of Deep Deterministic Policy Gradient (RS-DDPG) for mobile robots. To account for the poor convergence, the proposed approach first introduces an event-triggered reconfigurable actor–critic network framework for motion policy that adaptively changes its network structure to suppress the overestimation of action value. Then, the time convergence of the motion policy can be enhanced based on the value actions with minor valuation deviation. Afterwards, an adaptive reward mechanism is designed for reconfigurable networks to compensate for the lack of sample information. To deal with the problem of low learning efficiency, the research group developed a sample pretreatment method for the experience samples, which employs three novel techniques to improve the sample utilization, including a double experience memory buffer, a variable proportional sampling principle, and a similarity judgment mechanism. In extensive experiments, the proposed method outperforms the compared approaches.

In this paper, a novel learning-based motion planning approach, called event-triggered reconfigurable structure based on deep deterministic policy gradient (RS-DDPG) for mobile robots in unknown dynamic environments was proposed. Compared with other DRL-based motion planning approaches, the proposed approach has addressed the problems of poor convergence, insufficient sample information and low learning efficiency.

The first author of this paper is Sun Huihui, PhD student of the College of Technology, and corresponding authors are associate professor Hu Chunhe and professor Zhang Junguo. The unit of the first signature is Beijing Forestry University.

This work was supported by the National Natural Science Foundation of China (No. 61703047), and the Science and Technology Project of Hebei Education Department, China (No. QN2021312).

Paper link: https://www.sciencedirect.com/science/article/pii/S0952197623003810?via%3Dihub