Reproduce the results

[missTL]

Why I reproduce the results of subset A are so different from what is reported in the paper？

[Franpin]

Thanks for the answer, I'm going to check this question and can you show me your environment and the result that you tested with the weights we provided.

[missTL]

With the weights you guys provided, everything works fine. I didn't modify the source code Package Version

---

absl-py 2.1.0 addict 2.4.0 anyio 4.5.0 argon2-cffi 23.1.0 argon2-cffi-bindings 21.2.0 arrow 1.3.0 asttokens 2.4.1 async-lru 2.0.4 attrs 24.2.0 babel 2.16.0 backcall 0.2.0 beautifulsoup4 4.12.3 black 24.8.0 bleach 6.1.0 cachetools 5.5.0 certifi 2024.8.30 cffi 1.17.1 chardet 5.2.0 charset-normalizer 3.3.2 click 8.1.7 colorama 0.4.6 comm 0.2.2 contourpy 1.1.1 cycler 0.12.1 debugpy 1.8.6 decorator 5.1.1 defusedxml 0.7.1 descartes 1.1.0 exceptiongroup 1.2.2 executing 2.1.0 fastjsonschema 2.20.0 fire 0.7.0 flake8 7.1.1 fonttools 4.54.1 fqdn 1.5.1 google-auth 2.35.0 google-auth-oauthlib 1.0.0 grpcio 1.66.2 h11 0.14.0 httpcore 1.0.6 httpx 0.27.2 idna 3.10 imageio 2.35.1 importlib_metadata 8.5.0 importlib_resources 6.4.5 iniconfig 2.0.0 ipykernel 6.29.5 ipython 8.12.3 ipywidgets 8.1.5 iso3166 2.1.1 isoduration 20.11.0 jedi 0.19.1 Jinja2 3.1.4 joblib 1.4.2 json5 0.9.25 jsonpointer 3.0.0 jsonschema 4.23.0 jsonschema-specifications 2023.12.1 jupyter 1.1.1 jupyter_client 8.6.3 jupyter-console 6.6.3 jupyter_core 5.7.2 jupyter-events 0.10.0 jupyter-lsp 2.2.5 jupyter_server 2.14.2 jupyter_server_terminals 0.5.3 jupyterlab 4.2.5 jupyterlab_pygments 0.3.0 jupyterlab_server 2.27.3 jupyterlab_widgets 3.0.13 kiwisolver 1.4.7 llvmlite 0.36.0 lyft-dataset-sdk 0.0.8 Markdown 3.7 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.5.3 matplotlib-inline 0.1.7 mccabe 0.7.0 mdurl 0.1.2 mistune 3.0.2 mmcls 0.25.0 mmcv-full 1.6.0 mmdet 2.24.0 mmdet3d 1.0.0rc5 mmsegmentation 0.29.1 model-index 0.1.11 mypy-extensions 1.0.0 nbclient 0.10.0 nbconvert 7.16.4 nbformat 5.10.4 nest-asyncio 1.6.0 networkx 2.2 ninja 1.11.1.1 notebook 7.2.2 notebook_shim 0.2.4 numba 0.53.0 numpy 1.22.4 nuscenes-devkit 1.1.11 oauthlib 3.2.2 opencv-python 4.10.0.84 opendatalab 0.0.10 openlanev2 2.1.0 openmim 0.3.9 openxlab 0.0.11 ordered-set 4.1.0 ortools 9.2.9972 overrides 7.7.0 packaging 24.1 pandas 2.0.3 pandocfilters 1.5.1 parso 0.8.4 pathspec 0.12.1 pexpect 4.9.0 pickleshare 0.7.5 pillow 10.4.0 pip 24.2 pkgutil_resolve_name 1.3.10 platformdirs 4.3.6 plotly 5.24.1 pluggy 1.5.0 plyfile 1.0.3 prettytable 3.11.0 prometheus_client 0.21.0 prompt_toolkit 3.0.48 protobuf 5.28.2 psutil 6.0.0 ptyprocess 0.7.0 pure_eval 0.2.3 pyasn1 0.6.1 pyasn1_modules 0.4.1 pycocotools 2.0.7 pycodestyle 2.12.1 pycparser 2.22 pycryptodome 3.21.0 pyflakes 3.2.0 Pygments 2.18.0 pyparsing 3.1.4 pyquaternion 0.9.9 pytest 8.3.3 python-dateutil 2.9.0.post0 python-json-logger 2.0.7 pytz 2024.2 PyWavelets 1.4.1 PyYAML 6.0.2 pyzmq 26.2.0 referencing 0.35.1 requests 2.32.3 requests-oauthlib 2.0.0 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rich 13.9.2 rpds-py 0.20.0 rsa 4.9 scikit-image 0.19.3 scikit-learn 1.3.2 scipy 1.8.0 Send2Trash 1.8.3 setuptools 59.5.0 Shapely 1.8.5.post1 similaritymeasures 0.6.0 six 1.16.0 sniffio 1.3.1 soupsieve 2.6 stack-data 0.6.3 tabulate 0.9.0 tenacity 9.0.0 tensorboard 2.14.0 tensorboard-data-server 0.7.2 termcolor 2.4.0 terminado 0.18.1 terminaltables 3.1.10 threadpoolctl 3.5.0 tifffile 2023.7.10 tinycss2 1.3.0 tomli 2.0.2 torch 1.9.1+cu111 torchvision 0.10.1+cu111 tornado 6.4.1 tqdm 4.66.5 traitlets 5.14.3 trimesh 2.35.39 types-python-dateutil 2.9.0.20241003 typing_extensions 4.12.2 tzdata 2024.2 uri-template 1.3.0 urllib3 2.2.3 wcwidth 0.2.13 webcolors 24.8.0 webencodings 0.5.1 websocket-client 1.8.0 Werkzeug 3.0.4 wheel 0.44.0 widgetsnbextension 4.0.13 yapf 0.40.1 zipp 3.20.2

[Franpin]

Your mmdet3d and mmcv versions are inconsistent with mine, I updated requirement.txt, you can take a look at their version.