search

PaddlePaddle / PaddleSeg

Easy-to-use image segmentation library with awesome pre-trained model zoo, supporting wide-range of practical tasks in Semantic Segmentation, Interactive Segmentation, Panoptic Segmentation, Image Matting, 3D Segmentation, etc.
https://arxiv.org/abs/2101.06175
Apache License 2.0
8.62k stars 1.68k forks source link

复现全景图像分割出现配置文件问题，No lr_scheduler specified in the configuration file. #3820

Open yfq512 opened 4 days ago

yfq512 commented 4 days ago

问题确认 Search before asking

Bug描述 Describe the Bug

未改动任何代码和文件，使用自定义图片进行测试，测试“预测”部分报错：No lr_scheduler specified in the configuration file.

python -m paddle.distributed.launch predict.py --config configs/panoptic_deeplab/panoptic_deeplab_resnet50_os32_cityscapes_1025x513_bs8_90k_lr00005.yml --model_path model.pdparams --image_path imgs/ --save_dir outs22/
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
LAUNCH INFO 2024-10-09 02:29:14,745 -----------  Configuration  ----------------------
LAUNCH INFO 2024-10-09 02:29:14,745 auto_parallel_config: None
LAUNCH INFO 2024-10-09 02:29:14,745 auto_tuner_json: None
LAUNCH INFO 2024-10-09 02:29:14,745 devices: None
LAUNCH INFO 2024-10-09 02:29:14,745 elastic_level: -1
LAUNCH INFO 2024-10-09 02:29:14,745 elastic_timeout: 30
LAUNCH INFO 2024-10-09 02:29:14,746 enable_gpu_log: True
LAUNCH INFO 2024-10-09 02:29:14,746 gloo_port: 6767
LAUNCH INFO 2024-10-09 02:29:14,746 host: None
LAUNCH INFO 2024-10-09 02:29:14,746 ips: None
LAUNCH INFO 2024-10-09 02:29:14,746 job_id: default
LAUNCH INFO 2024-10-09 02:29:14,746 legacy: False
LAUNCH INFO 2024-10-09 02:29:14,746 log_dir: log
LAUNCH INFO 2024-10-09 02:29:14,746 log_level: INFO
LAUNCH INFO 2024-10-09 02:29:14,746 log_overwrite: False
LAUNCH INFO 2024-10-09 02:29:14,746 master: None
LAUNCH INFO 2024-10-09 02:29:14,746 max_restart: 3
LAUNCH INFO 2024-10-09 02:29:14,746 nnodes: 1
LAUNCH INFO 2024-10-09 02:29:14,746 nproc_per_node: None
LAUNCH INFO 2024-10-09 02:29:14,746 rank: -1
LAUNCH INFO 2024-10-09 02:29:14,746 run_mode: collective
LAUNCH INFO 2024-10-09 02:29:14,746 server_num: None
LAUNCH INFO 2024-10-09 02:29:14,746 servers:
LAUNCH INFO 2024-10-09 02:29:14,746 sort_ip: False
LAUNCH INFO 2024-10-09 02:29:14,746 start_port: 6070
LAUNCH INFO 2024-10-09 02:29:14,746 trainer_num: None
LAUNCH INFO 2024-10-09 02:29:14,746 trainers:
LAUNCH INFO 2024-10-09 02:29:14,746 training_script: predict.py
LAUNCH INFO 2024-10-09 02:29:14,746 training_script_args: ['--config', 'configs/panoptic_deeplab/panoptic_deeplab_resnet50_os32_cityscapes_1025x513_bs8_90k_lr00005.yml', '--model_path', 'model.pdparams', '--image_path', 'imgs/', '--save_dir', 'outs22/']
LAUNCH INFO 2024-10-09 02:29:14,746 with_gloo: 1
LAUNCH INFO 2024-10-09 02:29:14,746 --------------------------------------------------
LAUNCH INFO 2024-10-09 02:29:14,747 Job: default, mode collective, replicas 1[1:1], elastic False
LAUNCH INFO 2024-10-09 02:29:14,748 Run Pod: jcdgqp, replicas 1, status ready
LAUNCH INFO 2024-10-09 02:29:14,766 Watching Pod: jcdgqp, replicas 1, status running
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/paddleseg/cvlibs/config_checker.py", line 39, in apply_rule
    rule.apply(cfg, self.allow_update)
  File "/usr/local/lib/python3.10/dist-packages/paddleseg/cvlibs/config_checker.py", line 63, in apply
    self.check_and_correct(cfg)
  File "/usr/local/lib/python3.10/dist-packages/paddleseg/cvlibs/config_checker.py", line 73, in check_and_correct
    assert i in cfg.dic, \
AssertionError: No lr_scheduler specified in the configuration file.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/docker/yfq/paddle-test/PaddleSeg/contrib/PanopticDeepLab/predict.py", line 146, in <module>
    main(args)
  File "/data/docker/yfq/paddle-test/PaddleSeg/contrib/PanopticDeepLab/predict.py", line 111, in main
    cfg = Config(args.cfg)
  File "/usr/local/lib/python3.10/dist-packages/paddleseg/cvlibs/config.py", line 83, in __init__
    checker.apply_all_rules(self)
  File "/usr/local/lib/python3.10/dist-packages/paddleseg/cvlibs/config_checker.py", line 50, in apply_all_rules
    self.apply_rule(i, cfg)
  File "/usr/local/lib/python3.10/dist-packages/paddleseg/cvlibs/config_checker.py", line 41, in apply_rule
    raise RuntimeError(
RuntimeError: Sanity check on the configuration file has failed. There should be some problems with your config file. Please check it carefully.
The failed rule is DefaultPrimaryRule, and the error message is:
No lr_scheduler specified in the configuration file.
LAUNCH INFO 2024-10-09 02:29:17,770 Pod failed
LAUNCH ERROR 2024-10-09 02:29:17,771 Container failed !!!
Container rank 0 status failed cmd ['/usr/bin/python', '-u', 'predict.py', '--config', 'configs/panoptic_deeplab/panoptic_deeplab_resnet50_os32_cityscapes_1025x513_bs8_90k_lr00005.yml', '--model_path', 'model.pdparams', '--image_path', 'imgs/', '--save_dir', 'outs22/'] code 1 log log/workerlog.0
env {'NV_LIBCUBLAS_VERSION': '12.0.1.189-1', 'NVIDIA_VISIBLE_DEVICES': '0', 'NV_NVML_DEV_VERSION': '12.0.76-1', 'NV_CUDNN_PACKAGE_NAME': 'libcudnn8', 'GREP_COLOR': '1;31', 'NV_LIBNCCL_DEV_PACKAGE': 'libnccl-dev=2.17.1-1+cuda12.0', 'NV_LIBNCCL_DEV_PACKAGE_VERSION': '2.17.1-1', 'HOSTNAME': 'f4e7f826f6b8', 'LANGUAGE': 'en_US.UTF-8', 'NVIDIA_REQUIRE_CUDA': 'cuda>=12.0 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471', 'NV_LIBCUBLAS_DEV_PACKAGE': 'libcublas-dev-12-0=12.0.1.189-1', 'NV_NVTX_VERSION': '12.0.76-1', 'NV_CUDA_CUDART_DEV_VERSION': '12.0.107-1', 'NV_LIBCUSPARSE_VERSION': '12.0.0.76-1', 'NV_LIBNPP_VERSION': '12.0.0.30-1', 'NCCL_VERSION': '2.17.1-1', 'PWD': '/data/docker/yfq/paddle-test/PaddleSeg/contrib/PanopticDeepLab', 'NV_CUDNN_PACKAGE': 'libcudnn8=8.8.0.121-1+cuda12.0', 'NVIDIA_DRIVER_CAPABILITIES': 'compute,utility', 'WITH_AVX': 'ON', 'NV_NVPROF_DEV_PACKAGE': 'cuda-nvprof-12-0=12.0.90-1', 'NV_LIBNPP_PACKAGE': 'libnpp-12-0=12.0.0.30-1', 'NV_LIBNCCL_DEV_PACKAGE_NAME': 'libnccl-dev', 'GREP_OPTIONS': '--color=auto', 'NV_LIBCUBLAS_DEV_VERSION': '12.0.1.189-1', 'NVIDIA_PRODUCT_NAME': 'CUDA', 'NV_LIBCUBLAS_DEV_PACKAGE_NAME': 'libcublas-dev-12-0', 'NV_CUDA_CUDART_VERSION': '12.0.107-1', 'HOME': '/root', 'LANG': 'en_US.UTF-8', 'NVIDIA_CUDA_END_OF_LIFE': '1', 'CUDA_VERSION': '12.0.0', 'NV_LIBCUBLAS_PACKAGE': 'libcublas-12-0=12.0.1.189-1', 'NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE': 'cuda-nsight-compute-12-0=12.0.0-1', 'CLICOLOR': '1', 'NV_LIBNPP_DEV_PACKAGE': 'libnpp-dev-12-0=12.0.0.30-1', 'GOROOT': '/usr/local/go', 'NV_LIBCUBLAS_PACKAGE_NAME': 'libcublas-12-0', 'NV_LIBNPP_DEV_VERSION': '12.0.0.30-1', 'WITH_GPU': 'ON', 'TERM': 'xterm', 'NV_LIBCUSPARSE_DEV_VERSION': '12.0.0.76-1', 'LIBRARY_PATH': '/usr/local/cuda/lib64/stubs', 'NV_CUDNN_VERSION': '8.8.0.121', 'SHLVL': '1', 'NV_CUDA_LIB_VERSION': '12.0.0-1', 'NVARCH': 'x86_64', 'CUDNN_VERSION': '8.9.1', 'NV_CUDNN_PACKAGE_DEV': 'libcudnn8-dev=8.8.0.121-1+cuda12.0', 'NV_CUDA_COMPAT_PACKAGE': 'cuda-compat-12-0', 'NV_LIBNCCL_PACKAGE': 'libnccl2=2.17.1-1+cuda12.0', 'LD_LIBRARY_PATH': '/usr/local/lib/python3.10/dist-packages/cv2/../../lib64:/usr/local/TensorRT-8.6.1.6/lib:/usr/local/cuda-12.0/targets/x86_64-linux/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64', 'PS1': '\\[\\033[1;33m\\]λ \\[\\033[1;37m\\]\\h \\[\\033[1;32m\\]\\w \\[\\033[0m\\]', 'NV_CUDA_NSIGHT_COMPUTE_VERSION': '12.0.0-1', 'NV_NVPROF_VERSION': '12.0.90-1', 'LC_ALL': 'en_US.UTF-8', 'PATH': '/home/cmake-3.18.0-Linux-x86_64/bin:/usr/local/gcc-12.1/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin:/root/gopath/bin', 'NV_LIBNCCL_PACKAGE_NAME': 'libnccl2', 'NV_LIBNCCL_PACKAGE_VERSION': '2.17.1-1', 'DEBIAN_FRONTEND': 'noninteractive', 'OLDPWD': '/data/docker/yfq/paddle-test/PaddleSeg', 'GOPATH': '/root/gopath', '_': '/usr/bin/python', 'CUSTOM_DEVICE_ROOT': '', 'OMP_NUM_THREADS': '1', 'QT_QPA_PLATFORM_PLUGIN_PATH': '/usr/local/lib/python3.10/dist-packages/cv2/qt/plugins', 'QT_QPA_FONTDIR': '/usr/local/lib/python3.10/dist-packages/cv2/qt/fonts', 'POD_NAME': 'jcdgqp', 'PADDLE_MASTER': '192.168.32.2:44748', 'PADDLE_GLOBAL_SIZE': '1', 'PADDLE_LOCAL_SIZE': '1', 'PADDLE_GLOBAL_RANK': '0', 'PADDLE_LOCAL_RANK': '0', 'PADDLE_NNODES': '1', 'PADDLE_CURRENT_ENDPOINT': '192.168.32.2:44749', 'PADDLE_TRAINER_ID': '0', 'PADDLE_TRAINERS_NUM': '1', 'PADDLE_RANK_IN_NODE': '0', 'PADDLE_TRAINER_ENDPOINTS': '192.168.32.2:44749', 'FLAGS_selected_gpus': '0', 'PADDLE_LOG_DIR': '/data/docker/yfq/paddle-test/PaddleSeg/contrib/PanopticDeepLab/log'}
LAUNCH INFO 2024-10-09 02:29:17,771 ------------------------- ERROR LOG DETAIL -------------------------
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/paddleseg/cvlibs/config_checker.py", line 39, in apply_rule
    rule.apply(cfg, self.allow_update)
  File "/usr/local/lib/python3.10/dist-packages/paddleseg/cvlibs/config_checker.py", line 63, in apply
    self.check_and_correct(cfg)
  File "/usr/local/lib/python3.10/dist-packages/paddleseg/cvlibs/config_checker.py", line 73, in check_and_correct
    assert i in cfg.dic, \
AssertionError: No lr_scheduler specified in the configuration file.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/docker/yfq/paddle-test/PaddleSeg/contrib/PanopticDeepLab/predict.py", line 146, in <module>
    main(args)
  File "/data/docker/yfq/paddle-test/PaddleSeg/contrib/PanopticDeepLab/predict.py", line 111, in main
    cfg = Config(args.cfg)
  File "/usr/local/lib/python3.10/dist-packages/paddleseg/cvlibs/config.py", line 83, in __init__
    checker.apply_all_rules(self)
  File "/usr/local/lib/python3.10/dist-packages/paddleseg/cvlibs/config_checker.py", line 50, in apply_all_rules
    self.apply_rule(i, cfg)
  File "/usr/local/lib/python3.10/dist-packages/paddleseg/cvlibs/config_checker.py", line 41, in apply_rule
    raise RuntimeError(
RuntimeError: Sanity check on the configuration file has failed. There should be some problems with your config file. Please check it carefully.
The failed rule is DefaultPrimaryRule, and the error message is:
No lr_scheduler specified in the configuration file.
LAUNCH INFO 2024-10-09 02:29:17,771 Exit code 1

复现环境 Environment

使用的是官方提供的镜像 registry.baidubce.com/paddlepaddle/paddle:2.6.1-gpu-cuda12.0-cudnn8.9-trt8.6，并pip install paddleseg==2.8.0

Bug描述确认 Bug description confirmation

是否愿意提交PR？ Are you willing to submit a PR?

Bobholamovic commented 3 days ago

你好，PanopticDeepLab在新版本paddleseg尚未经过测试，请尝试使用paddleseg 2.6以及对应的release/2.6分支的PanopticDeepLab～