search

weiyithu / SurroundOcc

[ICCV 2023] SurroundOcc: Multi-camera 3D Occupancy Prediction for Autonomous Driving
Apache License 2.0
777 stars 95 forks source link

Problems about inference on template data #103

Open Benson722 opened 6 months ago

Benson722 commented 6 months ago

Thank you for your job!

When I try the inference code, some problems occur:

Code: ./tools/dist_inference.sh ./projects/configs/surroundocc/surroundocc_inference.py ./pretrainmodel/surroundocc.pth 1

Terminal:

(surroundocc) zhangyiqing@inin:~/SurroundOcc$ ./tools/dist_inference.sh ./projects/configs/surroundocc/surroundocc_inference.py ./pretrainmodel/surroundocc.pth 1
/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/distributed/launch.py:186: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions

  FutureWarning,
projects.mmdet3d_plugin
/home/zhangyiqing/SurroundOcc/projects/mmdet3d_plugin/surroundocc/modules/custom_base_transformer_layer.py:96: UserWarning: The arguments `feedforward_channels` in BaseTransformerLayer has been deprecated, now you should set `feedforward_channels` and other FFN related arguments to a dict named `ffn_cfgs`.
  f'The arguments `{ori_name}` in BaseTransformerLayer '
/home/zhangyiqing/SurroundOcc/projects/mmdet3d_plugin/surroundocc/modules/custom_base_transformer_layer.py:96: UserWarning: The arguments `ffn_dropout` in BaseTransformerLayer has been deprecated, now you should set `ffn_drop` and other FFN related arguments to a dict named `ffn_cfgs`.
  f'The arguments `{ori_name}` in BaseTransformerLayer '
/home/zhangyiqing/SurroundOcc/projects/mmdet3d_plugin/surroundocc/modules/custom_base_transformer_layer.py:96: UserWarning: The arguments `ffn_num_fcs` in BaseTransformerLayer has been deprecated, now you should set `num_fcs` and other FFN related arguments to a dict named `ffn_cfgs`.
  f'The arguments `{ori_name}` in BaseTransformerLayer '
/home/zhangyiqing/SurroundOcc/projects/mmdet3d_plugin/surroundocc/modules/custom_base_transformer_layer.py:96: UserWarning: The arguments `embed_dims` in BaseTransformerLayer has been deprecated, now you should set `embed_dims` and other FFN related arguments to a dict named `ffn_cfgs`.
  f'The arguments `{ori_name}` in BaseTransformerLayer '
load checkpoint from local path: ./pretrainmodel/surroundocc.pth
2024-03-08 17:01:10,581 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.0.conv2 is upgraded to version 2.
2024-03-08 17:01:10,588 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.1.conv2 is upgraded to version 2.
2024-03-08 17:01:10,592 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.2.conv2 is upgraded to version 2.
2024-03-08 17:01:10,597 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.3.conv2 is upgraded to version 2.
2024-03-08 17:01:10,601 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.4.conv2 is upgraded to version 2.
2024-03-08 17:01:10,605 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.5.conv2 is upgraded to version 2.
2024-03-08 17:01:10,610 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.6.conv2 is upgraded to version 2.
2024-03-08 17:01:10,614 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.7.conv2 is upgraded to version 2.
2024-03-08 17:01:10,618 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.8.conv2 is upgraded to version 2.
2024-03-08 17:01:10,623 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.9.conv2 is upgraded to version 2.
2024-03-08 17:01:10,625 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.10.conv2 is upgraded to version 2.
2024-03-08 17:01:10,627 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.11.conv2 is upgraded to version 2.
2024-03-08 17:01:10,629 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.12.conv2 is upgraded to version 2.
2024-03-08 17:01:10,631 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.13.conv2 is upgraded to version 2.
2024-03-08 17:01:10,634 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.14.conv2 is upgraded to version 2.
2024-03-08 17:01:10,636 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.15.conv2 is upgraded to version 2.
2024-03-08 17:01:10,638 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.16.conv2 is upgraded to version 2.
2024-03-08 17:01:10,640 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.17.conv2 is upgraded to version 2.
2024-03-08 17:01:10,642 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.18.conv2 is upgraded to version 2.
2024-03-08 17:01:10,644 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.19.conv2 is upgraded to version 2.
2024-03-08 17:01:10,647 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.20.conv2 is upgraded to version 2.
2024-03-08 17:01:10,649 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.21.conv2 is upgraded to version 2.
2024-03-08 17:01:10,651 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.22.conv2 is upgraded to version 2.
2024-03-08 17:01:10,654 - root - INFO - ModulatedDeformConvPack img_backbone.layer4.0.conv2 is upgraded to version 2.
2024-03-08 17:01:10,659 - root - INFO - ModulatedDeformConvPack img_backbone.layer4.1.conv2 is upgraded to version 2.
2024-03-08 17:01:10,662 - root - INFO - ModulatedDeformConvPack img_backbone.layer4.2.conv2 is upgraded to version 2.
[                                                  ] 0/227, elapsed: 0s, ETA:Traceback (most recent call last):
  File "./tools/test.py", line 269, in <module>
    main()
  File "./tools/test.py", line 238, in main
    args.gpu_collect, args.is_vis)
  File "/home/zhangyiqing/SurroundOcc/projects/mmdet3d_plugin/surroundocc/apis/test.py", line 73, in custom_multi_gpu_test
    for i, data in enumerate(data_loader):
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
    return self._process_data(data)
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
    data.reraise()
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/_utils.py", line 434, in reraise
    raise exception
FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/zhangyiqing/SurroundOcc/projects/mmdet3d_plugin/datasets/nuscenes_occupancy_dataset.py", line 129, in __getitem__
    return self.prepare_test_data(idx)
  File "/home/zhangyiqing/mmdetection3d/mmdet3d/datasets/custom_3d.py", line 172, in prepare_test_data
    example = self.pipeline(input_dict)
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/mmdet/datasets/pipelines/compose.py", line 40, in __call__
    data = t(data)
  File "/home/zhangyiqing/mmdetection3d/mmdet3d/datasets/pipelines/loading.py", line 47, in __call__
    [mmcv.imread(name, self.color_type) for name in filename], axis=-1)
  File "/home/zhangyiqing/mmdetection3d/mmdet3d/datasets/pipelines/loading.py", line 47, in <listcomp>
    [mmcv.imread(name, self.color_type) for name in filename], axis=-1)
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/mmcv/image/io.py", line 177, in imread
    f'img file does not exist: {img_or_path}')
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/mmcv/utils/path.py", line 23, in check_file_exist
    raise FileNotFoundError(msg_tmpl.format(filename))
FileNotFoundError: img file does not exist: ./in_the_wild/clip/0/000350.jpg

ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 3790785) of binary: /home/zhangyiqing/miniconda3/envs/surroundocc/bin/python
Traceback (most recent call last):
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/distributed/run.py", line 713, in run
    )(*cmd_args)
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/zhangyiqing/miniconda3/envs/surroundocc/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 261, in launch_agent
    failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
./tools/test.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-03-08_17:01:21
  host      : inin
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 3790785)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
==============================================

Look forward to your reply :) Thank you!

Benson722 commented 6 months ago

The path of document is:

(surroundocc) zhangyiqing@inin:~/SurroundOcc$ tree -d
.
├── assets
├── data
│   └── in_the_wild
│       └── clip
│           ├── 0
│           ├── 4
│           ├── 5
│           ├── 6
│           ├── 7
│           └── 8
├── docs
├── extensions
│   └── chamfer_dist
│       ├── build
│       │   ├── bdist.linux-x86_64
│       │   ├── lib.linux-x86_64-cpython-37
│       │   └── temp.linux-x86_64-cpython-37
│       ├── chamfer.egg-info
│       └── dist
├── pretrainmodel
├── projects
│   ├── configs
│   │   ├── _base_
│   │   │   ├── datasets
│   │   │   ├── models
│   │   │   └── schedules
│   │   ├── datasets
│   │   └── surroundocc
│   ├── mmdet3d_plugin
│   │   ├── core
│   │   │   ├── bbox
│   │   │   │   ├── assigners
│   │   │   │   │   └── __pycache__
│   │   │   │   ├── coders
│   │   │   │   │   └── __pycache__
│   │   │   │   ├── match_costs
│   │   │   │   │   └── __pycache__
│   │   │   │   └── __pycache__
│   │   │   └── evaluation
│   │   │       └── __pycache__
│   │   ├── datasets
│   │   │   ├── pipelines
│   │   │   │   └── __pycache__
│   │   │   ├── __pycache__
│   │   │   └── samplers
│   │   │       └── __pycache__
│   │   ├── models
│   │   │   ├── backbones
│   │   │   │   └── __pycache__
│   │   │   ├── hooks
│   │   │   ├── opt
│   │   │   │   └── __pycache__
│   │   │   └── utils
│   │   │       └── __pycache__
│   │   ├── __pycache__
│   │   └── surroundocc
│   │       ├── apis
│   │       │   └── __pycache__
│   │       ├── dense_heads
│   │       │   └── __pycache__
│   │       ├── detectors
│   │       │   └── __pycache__
│   │       ├── hooks
│   │       │   └── __pycache__
│   │       ├── loss
│   │       │   └── __pycache__
│   │       ├── modules
│   │       │   └── __pycache__
│   │       └── __pycache__
│   └── __pycache__
└── tools
    ├── analysis_tools
    ├── data_converter
    │   └── __pycache__
    ├── fp16
    ├── generate_occupancy_nuscenes
    ├── generate_occupancy_with_own_data
    ├── misc
    └── model_converters

79 directories
DeepakKarishetti commented 5 months ago

I had to move in_the_wild dir one folder up and inference_infos.pkl in the data folder to run using the pretrained model

Rogersljs commented 1 month ago

Hello, have you used surroundocc for training? I used the nuscenes-mini dataset and created the dataset pkl file in tools/create-data.py, but when training, an error message [Errno 2] No such file or directory: './data/nuscenes_occ/samples/n008-2018-08-27-11-48-51-0400__LIDAR_TOP__1535385106451625.pcd.bin' appeared, but the file does exist. Can you help me?