About nuscenes V1.0-mini dataset custom by maptv2 converter

[Jenny0420]

File "/home/jonas/PyProject/MapTR/tools/maptrv2/custom_nusc_map_converter.py", line 731, in union_centerline paths = nx.all_simple_paths(pts_G, root, leaves) File "/home/jonas/PyProject/MapTR/venv/lib/python3.10/site-packages/networkx/algorithms/simple_paths.py", line 202, in all_simple_paths raise nx.NodeNotFound('target node %s not in graph' % target) networkx.exception.NodeNotFound: target node [(-1.514, 30.0), (2.908, 30.0), (8.48, -30.0), (11.714, -30.0), (6.259, 30.0)] not in graph

How can i do?

[xmfcx]

I have the exact same problem.

I'm using the full dataset, not mini though.

I'm sharing my python packages, I think they are all the correct version.

🖱️Click here to expand🔛

```console $ pip list Package Version Editable project location ------------------------- -------------- -------------------------------------- absl-py 2.1.0 addict 2.4.0 anyio 4.4.0 argcomplete 3.5.0 argon2-cffi 23.1.0 argon2-cffi-bindings 21.2.0 arrow 1.3.0 asttokens 2.4.1 async-lru 2.0.4 attrs 24.2.0 av 12.3.0 babel 2.16.0 backcall 0.2.0 beautifulsoup4 4.12.3 black 24.8.0 bleach 6.1.0 cachetools 5.5.0 certifi 2024.8.30 cffi 1.17.1 charset-normalizer 3.3.2 click 8.1.7 colorlog 6.8.2 comm 0.2.2 contourpy 1.1.1 cycler 0.12.1 debugpy 1.8.5 decorator 5.1.1 defusedxml 0.7.1 descartes 1.1.0 distlib 0.3.8 exceptiongroup 1.2.2 executing 2.1.0 fastjsonschema 2.20.0 filelock 3.16.0 fire 0.6.0 flake8 7.1.1 fonttools 4.53.1 fqdn 1.5.1 fsspec 2024.9.0 GeometricKernelAttention 1.0 grpcio 1.66.1 h11 0.14.0 httpcore 1.0.5 httpx 0.27.2 huggingface-hub 0.24.7 idna 3.8 imageio 2.35.1 importlib_metadata 8.5.0 importlib_resources 6.4.5 iniconfig 2.0.0 ipykernel 6.29.5 ipython 8.12.3 ipywidgets 8.1.5 isoduration 20.11.0 jedi 0.19.1 Jinja2 3.1.4 joblib 1.4.2 json5 0.9.25 jsonpointer 3.0.0 jsonschema 4.23.0 jsonschema-specifications 2023.12.1 jupyter 1.1.1 jupyter_client 8.6.2 jupyter-console 6.6.3 jupyter_core 5.7.2 jupyter-events 0.10.0 jupyter-lsp 2.2.5 jupyter_server 2.14.2 jupyter_server_terminals 0.5.3 jupyterlab 4.2.5 jupyterlab_pygments 0.3.0 jupyterlab_server 2.27.3 jupyterlab_widgets 3.0.13 kiwisolver 1.4.7 lazy_loader 0.4 llvmlite 0.31.0 lyft-dataset-sdk 0.0.8 Markdown 3.7 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.6.3 matplotlib-inline 0.1.7 mccabe 0.7.0 mdurl 0.1.2 mistune 3.0.2 mmcv-full 1.4.0 mmdet 2.14.0 mmdet3d 0.17.2 /home/mfc/projects/MapTR/mmdetection3d mmsegmentation 0.14.1 mpmath 1.3.0 mypy-extensions 1.0.0 nbclient 0.10.0 nbconvert 7.16.4 nbformat 5.10.4 nest-asyncio 1.6.0 networkx 2.2 notebook 7.2.2 notebook_shim 0.2.4 nox 2024.4.15 numba 0.48.0 numpy 1.19.5 nuscenes-devkit 1.1.9 nvidia-cublas-cu12 12.1.3.1 nvidia-cuda-cupti-cu12 12.1.105 nvidia-cuda-nvrtc-cu12 12.1.105 nvidia-cuda-runtime-cu12 12.1.105 nvidia-cudnn-cu12 9.1.0.70 nvidia-cufft-cu12 11.0.2.54 nvidia-curand-cu12 10.3.2.106 nvidia-cusolver-cu12 11.4.5.107 nvidia-cusparse-cu12 12.1.0.106 nvidia-nccl-cu12 2.20.5 nvidia-nvjitlink-cu12 12.6.68 nvidia-nvtx-cu12 12.1.105 opencv-python 4.10.0.84 overrides 7.7.0 packaging 24.1 pandas 1.4.4 pandocfilters 1.5.1 parso 0.8.4 pathspec 0.12.1 pexpect 4.9.0 pickleshare 0.7.5 pillow 10.4.0 pip 24.2 pkgutil_resolve_name 1.3.10 platformdirs 4.3.2 plotly 5.24.0 pluggy 1.5.0 plyfile 1.0.3 prettytable 3.11.0 prometheus_client 0.20.0 prompt_toolkit 3.0.47 protobuf 5.28.1 psutil 6.0.0 ptyprocess 0.7.0 pure_eval 0.2.3 pyarrow 17.0.0 pycocotools 2.0.7 pycodestyle 2.12.1 pycparser 2.22 pyflakes 3.2.0 Pygments 2.18.0 pyparsing 3.1.4 pyproj 3.5.0 pyquaternion 0.9.9 pytest 8.3.3 python-dateutil 2.9.0.post0 python-json-logger 2.0.7 pytz 2024.2 PyWavelets 1.4.1 PyYAML 6.0.2 pyzmq 26.2.0 referencing 0.35.1 requests 2.32.3 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rich 13.8.1 rpds-py 0.20.0 safetensors 0.4.5 scikit-image 0.19.3 scikit-learn 1.3.2 scipy 1.10.1 Send2Trash 1.8.3 setuptools 74.1.2 Shapely 1.8.5.post1 six 1.16.0 sniffio 1.3.1 soupsieve 2.6 stack-data 0.6.3 sympy 1.13.2 tenacity 9.0.0 tensorboard 2.17.1 tensorboard-data-server 0.7.2 termcolor 2.4.0 terminado 0.18.1 terminaltables 3.1.10 threadpoolctl 3.5.0 tifffile 2023.7.10 timm 1.0.9 tinycss2 1.3.0 tomli 2.0.1 torch 1.9.1+cu111 torchaudio 0.9.1 torchvision 0.10.1+cu111 tornado 6.4.1 tqdm 4.66.5 traitlets 5.14.3 trimesh 2.35.39 triton 3.0.0 types-python-dateutil 2.9.0.20240906 typing_extensions 4.12.2 tzdata 2024.1 uri-template 1.3.0 urllib3 2.2.3 virtualenv 20.26.4 wcwidth 0.2.13 webcolors 24.8.0 webencodings 0.5.1 websocket-client 1.8.0 Werkzeug 3.0.4 wheel 0.43.0 widgetsnbextension 4.0.13 yapf 0.40.2 zipp 3.20.1 ```

I've used pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html mmdet==2.14.0 "numpy<1.20.0" nuscenes-devkit shapely==1.8.5.post1 mmsegmentation==0.14.1 mmcv-full==1.4.0 timm "networkx<2.3,>=2.2" --force-reinstall command to make sure everything has the correct version.

Also removed av2 from the requirements.txt file since I will not use argoverse dataset.

But still, I get the same error:

(maptr) mfc@mfc-leo:~/projects/MapTR$ python tools/maptrv2/custom_nusc_map_converter.py --root-path ./data/nuscenes --out-dir ./data/nuscenes --extra-tag nuscenes --version v1.0 --canbus ./data
v1.0-trainval ./data/nuscenes
======
Loading NuScenes tables for version v1.0-trainval...
23 category,
8 attribute,
4 visibility,
64386 instance,
12 sensor,
10200 calibrated_sensor,
2631083 ego_pose,
68 log,
850 scene,
34149 sample,
2631083 sample_data,
1166187 sample_annotation,
4 map,
Done loading in 26.446 seconds.
======
Reverse indexing ...
Done reverse indexing in 5.7 seconds.
======
total scene num: 850
exist scene num: 850
train scene: 700, val scene: 150
[                                                  ] 0/34149, elapsed: 0s, ETA:Traceback (most recent call last):
  File "tools/maptrv2/custom_nusc_map_converter.py", line 928, in <module>
    nuscenes_data_prep(
  File "tools/maptrv2/custom_nusc_map_converter.py", line 869, in nuscenes_data_prep
    create_nuscenes_infos(
  File "tools/maptrv2/custom_nusc_map_converter.py", line 825, in create_nuscenes_infos
    train_nusc_infos, val_nusc_infos = _fill_trainval_infos(
  File "tools/maptrv2/custom_nusc_map_converter.py", line 340, in _fill_trainval_infos
    info = obtain_vectormap(nusc_maps, map_explorer, info, point_cloud_range)
  File "tools/maptrv2/custom_nusc_map_converter.py", line 371, in obtain_vectormap
    map_anns = vector_map.gen_vectorized_samples(lidar2global_translation, lidar2global_rotation)
  File "tools/maptrv2/custom_nusc_map_converter.py", line 442, in gen_vectorized_samples
    centerline_list = self.centerline_geoms_to_instances(centerline_geom)
  File "tools/maptrv2/custom_nusc_map_converter.py", line 672, in centerline_geoms_to_instances
    centerline_geoms_list,pts_G = self.union_centerline(geoms_dict)
  File "tools/maptrv2/custom_nusc_map_converter.py", line 743, in union_centerline
    paths = nx.all_simple_paths(pts_G, root, leaves)
  File "/home/mfc/miniconda3/envs/maptr/lib/python3.8/site-packages/networkx/algorithms/simple_paths.py", line 202, in all_simple_paths
    raise nx.NodeNotFound('target node %s not in graph' % target)
networkx.exception.NodeNotFound: target node [(15.0, 2.355), (15.0, -2.636), (4.167, -30.0), (-15.0, 15.813), (0.284, -30.0)] not in graph
(maptr) mfc@mfc-leo:~/projects/MapTR$ 

@LegendBC could you help please?

[cyn-liu]

@xmfcx The above error is caused by a mismatch version of networkx. I solve this error when I change version to networkx==3.1. You can try it

[cyn-liu]

@xmfcx I don't think we need to fully meet the version requirements of mmdet3d 0.17.2, otherwise it will cause conflicts with other packages version. Here are the version information in my environment, especially the following packages to pay attention to:

numba==0.48.0
numpy==1.23.5

Package                  Version      Location
------------------------ ------------ ------------------------------
absl-py                  2.0.0
addict                   2.4.0
albumentations           0.4.6
appdirs                  1.4.4
audioread                3.0.0
brotlipy                 0.7.0
cachetools               5.3.1
certifi                  2022.12.7
cffi                     1.15.0
charset-normalizer       2.0.4
colorama                 0.4.4
conda-content-trust      0+unknown
conda-package-handling   1.8.1
contourpy                1.1.1
cryptography             36.0.0
cycler                   0.12.1
decorator                5.1.1
descartes                1.1.0
dill                     0.3.7
filelock                 3.16.1
fire                     0.6.0
fonttools                4.43.1
fsspec                   2024.9.0
future                   0.18.3
GeometricKernelAttention 1.0
get-f0                   0.2.3
google-auth              2.23.3
google-auth-oauthlib     1.0.0
grpcio                   1.59.0
huggingface-hub          0.25.0
idna                     3.3
imageio                  2.31.5
imgaug                   0.4.0
importlib-metadata       6.8.0
importlib-resources      6.1.0
joblib                   1.2.0
kiwisolver               1.4.5
lazy_loader              0.3
librosa                  0.9.2
llvmlite                 0.31.0
lyft-dataset-sdk         0.0.8
Markdown                 3.5
MarkupSafe               2.1.3
matplotlib               3.5.3
mmcv-full                1.4.0
mmdet                    2.14.0
mmdet3d                  0.17.2       /workspace/MapTRv2/mmdetection3d
mmsegmentation           0.14.1
multiprocess             0.70.15
networkx                 3.1
numba                    0.48.0
numpy                    1.23.5
nuscenes-devkit          1.1.11
oauthlib                 3.2.2
opencv-python            4.8.1.78
opencv-python-headless   4.8.1.78
p-tqdm                   1.4.0
packaging                23.0
pandas                   2.0.3
pathos                   0.3.1
Pillow                   10.0.1
pip                      21.2.4
platformdirs             3.11.0
pluggy                   1.0.0
plyfile                  1.1
pooch                    1.6.0
pox                      0.3.3
ppft                     1.7.6.7
prettytable              3.11.0
protobuf                 4.24.4
pyasn1                   0.5.0
pyasn1-modules           0.3.0
pycocotools              2.0.7
pycosat                  0.6.3
pycparser                2.21
pyOpenSSL                22.0.0
pyparsing                3.1.1
pyquaternion             0.9.9
PySocks                  1.7.1
python-dateutil          2.8.2
pytz                     2024.2
PyWavelets               1.4.1
PyYAML                   6.0.1
qudida                   0.0.4
requests                 2.27.1
requests-oauthlib        1.3.1
resampy                  0.4.2
rsa                      4.9
ruamel.yaml              0.17.21
ruamel.yaml.clib         0.2.6
ruamel-yaml-conda        0.15.100
safetensors              0.4.5
scikit-image             0.21.0
scikit-learn             1.2.1
scipy                    1.10.0
setuptools               65.5.1
Shapely                  1.8.5
six                      1.16.0
soundfile                0.11.0
tensorboard              2.14.0
tensorboard-data-server  0.7.1
termcolor                2.4.0
terminaltables           3.1.10
threadpoolctl            3.1.0
tifffile                 2023.7.10
timm                     1.0.9
tomli                    2.0.1
toolz                    0.12.0
torch                    1.9.0+cu111
torchaudio               0.9.0
torchvision              0.10.0+cu111
tqdm                     4.63.0
trimesh                  2.35.39
typing_extensions        4.8.0
tzdata                   2024.1
urllib3                  1.26.8
wcwidth                  0.2.13
Werkzeug                 3.0.0
wheel                    0.38.1
yapf                     0.40.1
zipp                     3.11.0

[xmfcx]

Thanks @cyn-liu !

After running my previous:

pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html mmdet==2.14.0 "numpy<1.20.0" nuscenes-devkit shapely==1.8.5.post1 mmsegmentation==0.14.1 mmcv-full==1.4.0 timm "networkx<2.3,>=2.2"

I ran:

pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html mmdet==2.14.0 nuscenes-devkit shapely==1.8.5.post1 mmsegmentation==0.14.1 mmcv-full==1.4.0 timm numpy==1.23.5 networkx==3.1 numba==0.48.0

To comply with the versions you've shared.

I've got these errors but I ignored them like you've said.

I ran numpy==1.23.5 networkx==3.1 numba==0.48.0 separately to make sure they are installed.

Then https://github.com/hustvl/MapTR/blob/e03f097abef19e1ba3fed5f471a8d80fbfa0a064/docs/prepare_dataset.md?plain=1#L18 ran correctly.

[yshen47]

I wonder what does the prediction look like on v1.0 mini? I followed above setup and able to run everything. But the result is random prediction, lol. like the following.

[DarrenWong]

I wonder what does the prediction look like on v1.0 mini? I followed above setup and able to run everything. But the result is random prediction, lol. like the following.

Hi, I have similar results on Maptr2, do you solve it already? Mandy thanks

[xmfcx]

I had downloaded the entire dataset. For me, the predictions looked as they should. I didn't see outputs like yours. I did not test on mini though.

[DarrenWong]

Thanks for your suggestions and I will try again!

[cyn-liu]

Thanks for your suggestions and I will try again!

I used the nuscenes-mini dataset for MapTR v1 prediction visualization, and my results looks normal. I think you should pay attention to the following two aspects.

1. Check your running environment
2. Check custom annotation files your have generated

I use the following command to generate custom annotation files:

python tools/create_data.py nuscenes --root-path ./data/nuscenes --out-dir ./data/nuscenes --extra-tag nuscenes --version v1.0-mini --canbus ./data/nuscenes

I use the following command to visualize prediction:

cd /path/to/MapTR/
export PYTHONPATH="/path/to/MapTR/"
python tools/maptr/vis_pred.py projects/configs/maptr/maptr_tiny_r50_24e_t4.py ckpt/maptr_tiny_r50_24e.pth

All the visualization samples of mine will be saved in /path/to/MapTR/work_dirs/maptr_tiny_r50_24e_t4/vis_pred/.

[DarrenWong]

Thanks for your suggestions and I will try again!

I used the nuscenes-mini dataset for MapTR v1 prediction visualization, and my results looks normal. I think you should pay attention to the following two aspects.

1. Check your running environment
2. Check custom annotation files your have generated

I use the following command to generate custom annotation files:

python tools/create_data.py nuscenes --root-path ./data/nuscenes --out-dir ./data/nuscenes --extra-tag nuscenes --version v1.0-mini --canbus ./data/nuscenes

I use the following command to visualize prediction:

cd /path/to/MapTR/
export PYTHONPATH="/path/to/MapTR/"
python tools/maptr/vis_pred.py projects/configs/maptr/maptr_tiny_r50_24e_t4.py ckpt/maptr_tiny_r50_24e.pth

All the visualization samples of mine will be saved in /path/to/MapTR/work_dirs/maptr_tiny_r50_24e_t4/vis_pred/.

Thank you for your reply. I am using the maptv2, so might be I might also try v1 first.

[cyn-liu]

Thank you for your reply. I am using the maptv2, so might be I might also try v1 first.

I used the nuscenes-mini dataset for MapTR v2 prediction visualization, and my results looks also normal!

Notes: annotation generation of MapTRv2 is different from MapTR

[yshen47]

I wonder what does the prediction look like on v1.0 mini? I followed above setup and able to run everything. But the result is random prediction, lol. like the following.

Hi, I have similar results on Maptr2, do you solve it already? Mandy thanks

Sorry for the late reply! In my case, I loaded the wrong checkpoint weights. I previously mistakenly loaded only the resenet weight, but in fact you should directly load the actual model weight (even though both seems to be able to load with no error)

[DarrenWong]

many thanks, both! I will evaluate again step by steps