PaddlePaddle / Paddle

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice (『飞桨』核心框架,深度学习&机器学习高性能单机、分布式训练和跨平台部署)
http://www.paddlepaddle.org/
Apache License 2.0
22.09k stars 5.55k forks source link

No stack trace in paddle, may be caused by external reasons. #59983

Closed ZS-YANG closed 9 months ago

ZS-YANG commented 9 months ago

bug描述 Describe the Bug

Bug描述 Describe the Bug

复现步骤:

下载cityscapesScripts数据集,使用

python tools/data/convert_cityscapes.py

转换数据集,更改

configs/_base_/cityscapes.yml

开始训练

python tools/train.py        --config configs/deeplabv3/deeplabv3_resnet50_os8_cityscapes_1024x512_80k.yml        --save_interval 500        --do_eval         --save_dir output_pid

报错内容

--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
No stack trace in paddle, may be caused by external reasons.

----------------------
Error Message Summary:
----------------------
FatalError: `Segmentation fault` is detected by the operating system.
  [TimeInfo: *** Aborted at 1702453676 (unix time) try "date -d @1702453676" if you are using GNU date ***]
  [SignalInfo: *** SIGSEGV (@0x0) received by PID 35373 (TID 0x7f0a2f217700) from PID 0 ***]

Segmentation fault (core dumped)

复现环境 Environment

Linux paddlepaddle-gpu 0.0.0.post102 paddleseg 0.0.0.dev0
python 3.8 Cuda10.2

其他补充信息 Additional Supplementary Information

No response

ZS-YANG commented 9 months ago

https://github.com/PaddlePaddle/PaddleSeg/issues/3596

ZS-YANG commented 9 months ago

@JZ-LIANG 报错的有 configs/deeplabv3/deeplabv3_resnet50_os8_cityscapes_1024x512_80k.yml configs/unet/unet_cityscapes_1024x512_160k.yml configs/u2net/u2net_cityscapes_1024x512_160k.yml configs/unet_plusplus/unet_plusplus_cityscapes_1024x512_160k.yml

不报错的有 configs/deeplabv3p/deeplabv3p_resnet50_os8_cityscapes_1024x512_80k.yml configs/pp_liteseg/pp_liteseg_stdc2_cityscapes_1024x512_scale0.5_160k.yml configs/ocrnet/ocrnet_hrnetw18_cityscapes_1024x512_160k.yml configs/fcn/fcn_hrnetw18_cityscapes_1024x512_80k.yml configs/mobileseg/mobileseg_ghostnet_cityscapes_1024x512_80k.yml configs/u2net/u2netp_cityscapes_1024x512_160k.yml

JZ-LIANG commented 9 months ago

deeplabv3 套件更换数据集问题,部分数据集预处理 segment fault 报错,预处理逻辑存在问题

请使用自带数据集