AccidentGPT: Accident Analysis and Prevention from V2X Environmental Perception with Multi-modal Large Model

Lening Wang, Yilong Ren*, Han Jiang, Zhiyong Cui*, Haiyang Yu*    Pinlong Cai, Daocheng Fu
State Key Lab of Intelligent Transportation System
Beihang University, Beijing     Shanghai Artificial Intelligence Laboratory
Shanghai
Tianqi Wang                         Xuesong Wang                  Hanchu Zhou, Helai Huang                       Yinhai Wang
The University of Hong Kong
Hong Kong   Tongji University
Shanghai   Central South University
Changsha   University of Washington
Seattle
First author: leningwang@buaa.edu.cn *Corresponding author: yilongren@buaa.edu.cn; zhiyongc@buaa.edu.cn; hyyu@buaa.edu.cn
Code arXiv

AccidentGPT Video

Abstract

Traffic accidents, being a significant contributor to both human casualties and property damage, have long been a focal point of research for many scholars in the field of traffic safety. However, previous studies, whether focusing on static environmental assessments or dynamic driving analyses, as well as pre-accident predictions or post-accident rule analyses, have typically been conducted in isolation. There has been a lack of an effective framework for developing a comprehensive understanding and application of traffic safety. To address this gap, this paper introduces AccidentGPT, a comprehensive accident analysis and prevention multi-modal large model. AccidentGPT establishes a multi-modal information interaction framework grounded in multi-sensor perception, thereby enabling a holistic approach to accident analysis and prevention in the field of traffic safety. Specifically, our capabilities can be categorized as follows: for autonomous driving vehicles, we provide comprehensive environmental perception and understanding to control the vehicle and avoid collisions. For human-driven vehicles, we offer proactive long-range safety warnings and blind-spot alerts while also providing safety driving recommendations and behavioral norms through human-machine dialogue and interaction. Additionally, for traffic police and management agencies, our framework supports intelligent and real-time analysis of traffic safety, encompassing pedestrian, vehicles, roads, and the environment through collaborative perception from multiple vehicles and road testing devices. The system is also capable of providing a thorough analysis of accident causes and liability after vehicle collisions. Our framework stands as the first large model to integrate comprehensive scene understanding into traffic safety studies.

Paper

BibTeX

@article{wang2023accidentgpt,
  title={AccidentGPT: Accident Analysis and Prevention from V2X Environmental Perception with Multi-modal Large Model},
  author={Wang, Lening and Ren, Yilong and Jiang, Han and Cai, Pinlong and Fu, Daocheng and Wang, Tianqi and Cui, Zhiyong and Yu, Haiyang and Wang, Xuesong and Zhou, Hanchu and Huang, Helai and Wang, Yinhai},
  journal={arXiv preprint arXiv:2312.13156},
  year={2023}
}