No stack trace in paddle, may be caused by external reasons.

ZS-YANG commented 11 months ago

问题确认 Search before asking

[X] 我已经查询历史issue(包括open与closed)，没有发现相似的bug。I have searched the open and closed issues and found no similar bug report.

Bug描述 Describe the Bug

复现步骤:

下载cityscapesScripts数据集，使用

python tools/data/convert_cityscapes.py

转换数据集，更改

configs/_base_/cityscapes.yml

开始训练

python tools/train.py        --config configs/deeplabv3/deeplabv3_resnet50_os8_cityscapes_1024x512_80k.yml        --save_interval 500        --do_eval         --save_dir output_pid

报错内容

--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
No stack trace in paddle, may be caused by external reasons.

----------------------
Error Message Summary:
----------------------
FatalError: `Segmentation fault` is detected by the operating system.
  [TimeInfo: *** Aborted at 1702453676 (unix time) try "date -d @1702453676" if you are using GNU date ***]
  [SignalInfo: *** SIGSEGV (@0x0) received by PID 35373 (TID 0x7f0a2f217700) from PID 0 ***]

Segmentation fault (core dumped)

复现环境 Environment

Linux paddlepaddle-gpu 0.0.0.post102 paddleseg 0.0.0.dev0
python 3.8 Cuda10.2

Bug描述确认 Bug description confirmation

[X] 我确认已经提供了Bug复现步骤、代码改动说明、以及环境信息，确认问题是可以复现的。I confirm that the bug replication steps, code change instructions, and environment information have been provided, and the problem can be reproduced.

是否愿意提交PR？ Are you willing to submit a PR?

[X] 我愿意提交PR！I'd like to help by submitting a PR!

ZS-YANG commented 11 months ago

报错的有 configs/deeplabv3/deeplabv3_resnet50_os8_cityscapes_1024x512_80k.yml configs/unet/unet_cityscapes_1024x512_160k.yml configs/u2net/u2net_cityscapes_1024x512_160k.yml configs/unet_plusplus/unet_plusplus_cityscapes_1024x512_160k.yml

不报错的有 configs/deeplabv3p/deeplabv3p_resnet50_os8_cityscapes_1024x512_80k.yml configs/pp_liteseg/pp_liteseg_stdc2_cityscapes_1024x512_scale0.5_160k.yml configs/ocrnet/ocrnet_hrnetw18_cityscapes_1024x512_160k.yml configs/fcn/fcn_hrnetw18_cityscapes_1024x512_80k.yml configs/mobileseg/mobileseg_ghostnet_cityscapes_1024x512_80k.yml configs/u2net/u2netp_cityscapes_1024x512_160k.yml

ZS-YANG commented 11 months ago

@shiyutang 辛苦帮忙看下

PaddlePaddle / PaddleSeg