交通密集水域航行船舶较多，交通组成复杂，通航密度较大，多船会遇情况下由于船舶操纵性不同，难以准确预判他船动态轨迹与自身航线的冲突点，导致避碰策略失效。因此，提出多智能体深度强化学习下多目标船舶避碰决策方法。首先，基于人工势场法计算船舶的碰撞危险度，以确定存在碰撞风险的船舶；其次，基于船舶的状态、动作和奖励函数来构建船舶多智能体避碰决策系统，并将船舶的航行目标和安全性目标引入奖励函数设计中；最后，利用深度强化学习算法获得系统中的最优避碰策略。实验结果表明，该方法能够有效评估船舶碰撞风险，且避碰效果好，避碰路径短，具有实际应用价值。
There are many ships sailing in densely populated waters, with complex traffic composition and high navigation density. In situations where multiple ships encounter different maneuverability, it is difficult to accurately predict the conflict points between the dynamic trajectory of other ships and their own route, resulting in the failure of collision avoidance strategies. Therefore, a multi-objective ship collision avoidance decision-making method based on multi-agent deep reinforcement learning is proposed. Firstly, the collision risk of ships is calculated based on the artificial potential field method to determine the ships with collision risks; Secondly, based on the state, actions, and reward functions of the ship, a multi-agent collision avoidance decision-making system for the ship is constructed, and the navigation and safety objectives of the ship are introduced into the design of the reward function; Finally, the optimal collision avoidance strategy in the system is obtained using deep reinforcement learning algorithms. The experimental results show that this method can effectively evaluate the risk of ship collision, and has good collision avoidance effect, short collision avoidance path, and practical application value.
2025,47(23): 71-77 收稿日期：2025-5-8
DOI：10.3404/j.issn.1672-7649.2025.23.011
分类号：U675.96
作者简介：郭洪宇(1996-),男,博士,工程师,研究方向为人工智能决策技术、容器调度
参考文献：
[1] 廖诗管, 翁金贤. 基于贝叶斯时空log-logistic模型的船舶碰撞频率[J]. 中国航海, 2023, 46(1): 24-29+38.
LIAO S G, WENG J X. Ship collision frequency prediction with bayesian spatiotemporal log-logistic model[J]. Navigation of China, 2023, 46(1): 24-29+38.
[2] 杨琪森, 王慎执, 桑金楠, 等. 复杂开放水域下智能船舶路径规划与避障方法[J]. 计算机集成制造系统, 2022, 28(7): 2030-2040.
YANG Q S, WANG S Z, SANG J N, et al. Path planning and real-time obstacle avoidance methods of intelligent ships in complex open water environment[J]. Computer Integrated Manufacturing Systems, 2022, 28(7): 2030-2040.
[3] 宁君, 黄寓旸, 尤恽, 等. 基于混合粒子群算法的船舶避碰决策[J]. 大连海事大学学报, 2023, 49(1): 34-43.
NING J, HUANG Y Y, YOU Y, et al. Ship collision avoidance decision based on hybrid particle swarm algorithm[J]. Journal of Dalian Maritime University, 2023, 49(1): 34-43.
[4] 关巍, 王淼淼, 韩虎生, 等. 基于Dueling-DDQN的船舶智能避碰决策方法[J]. 大连海事大学学报, 2024, 50(4): 22-30.
GUAN W, WANG M M, HAN H S, et al. Intelligent collision avoidance decision-making method for ships based on Dueling DDQN[J]. Journal of Dalian Maritime University, 2024, 50(4): 22-30.
[5] 张伟龙, 单梁, 常路, 等. 基于改进DWA的多无人水面艇分布式避碰算法[J]. 控制与决策, 2023, 38(4): 951-962.
ZHANG W L, SHAN L, CHANG L, et al. Distributed collision avoidance algorithm for multiple unmanned surface vessels based on improved DWA[J]. Control and Decision, 2023, 38(4): 951-962.
[6] 谭智坤, 张隆辉, 刘正锋, 等. 融合改进动态窗口法与速度障碍法的无人船局部路径规划[J]. 船舶力学, 2023, 27(3): 311-322.
TAN Z K, ZHANG L H, LIU Z F, et al. Local path planning for USVs based on the fusion algorithm of improved dynamic window approach and velocity obstacle algorithm[J]. Journal of Ship Mechanics, 2023, 27(3): 311-322.
[7] 王庆禄, 吴冯国, 郑成辰, 等. 基于优化人工势场法的无人机航迹规划[J]. 系统工程与电子技术, 2023, 45(5): 1461-1468.
WANG Q L, WU F G, ZHENG C C, et al. UAV path planning based on optimized artificial potential field method[J]. Systems Engineering and Electronics, 2023, 45(5): 1461-1468.
[8] 郑维, 王昊, 王洪斌. 动态环境下基于自适应步长Informed-RRT*和人工势场法的机器人混合路径规划[J]. 计量学报, 2023, 44(1): 26-34.
ZHENG W, WANG H, WANG H B. Adaptive step size Informed-RRT*and artificial potential field algorithm for hybrid path planning of robot[J]. Acta Metrologica Sinica, 2023, 44(1): 26-34.
[9] 张智超, 张景峰, 杨栋, 等. 基于目标失效概率的桥梁船撞风险及防撞水准论证[J]. 公路交通科技, 2023, 40(2): 72-80.
ZHANG Z C, ZHANG J F, YANG D, et al. Demonstration of bridge-vessel collision risk and fortification criterion against vessel collision based on target failure probability[J]. Journal of Highway and Transportation Research and Development, 2023, 40(2): 72-80.
[10] 孔祥磊, 汪芳琴, 钟选明, 等. 具有方向约束的多智能体系统的反一致性研究[J]. 空间控制技术与应用, 2023, 49(1): 74-81.
KONG X L, WANG F Q, ZHONG X M, et al. Inverse consensus of multi-agent systems with directional constraints[J]. Aerospace Control and Application, 2023, 49(1): 74-81.
[11] 丁伟, 明振军, 王国新, 等. 基于多层次LSTM网络的多智能体攻防效能动态预测模型[J]. 兵工学报, 2023, 44(1): 176-192.
DING W, MING Z J, WANG G X, et al. Dynamic prediction model based on multi-level LSTM network for multi-agent attack and defense effectiveness[J]. Acta Armamentarii, 2023, 44(1): 176-192.
[12] 田维青, 彭雪飞, 王成军, 等. 基于深度神经网络的电厂跑冒滴漏智能识别方法研究[J]. 电子器件, 2024, 47(2): 524-529.
TIAN W Q, PENG X F, WANG C J, et al. Research on intelligent identification method of power plant leakage based on deep neural network[J]. Chinese Journal of Electron Devices, 2024, 47(2): 524-529.
[13] 杨茂桃, 梁爽, 易淼荣, 等. 基于深度神经网络的超声速隔离段湍流涡黏性系数辨识[J]. 航空动力学报, 2023, 38(2): 312-324.
YANG M T, LIANG S, YI M R, et al. Identification of turbulence eddy viscosity coefficient in supersonic isolation section based on deep neural network[J]. Journal of Aerospace Power, 2023, 38(2): 312-324.
[14] 武晨雨, 陶银罗, 曾九孙. 引入注意力机制时空深度神经网络的再热器温度偏差预测方法[J]. 中国测试, 2024, 50(1): 151-159+192.
WU C Y, TAO Y L, ZENG J S. Prediction method of reheater temperature deviation based on attention mechanism spatiotemporal deep neural network[J]. China Measurement & Test, 2024, 50(1): 151-159+192.
[15] 黄国良, 周毅, 郑坤, 等. 基于改进蚁群算法的全局船舶路径规划方法[J]. 船海工程, 2023, 52(2): 97-101+136.
HUANG G L, ZHOU Y, ZHENG K, et al. Ship path planning and collision avoidance based on improved ant colony algorithm[J]. Ship & Ocean Engineering, 2023, 52(2): 97-101+136.
