I cannot reproduce your performance results.

[jouhy]

We trained the dsgn2 model, but there is a significant difference in performance between training and evaluation.

Our system construction is as follows.

Package                 Version                        Editable project location
----------------------- ------------------------------ ------------------------------
absl-py                 1.3.0
addict                  2.4.0
apex                    0.1
appdirs                 1.4.4
argon2-cffi             21.3.0
argon2-cffi-bindings    21.2.0
asttokens               2.1.0
astunparse              1.6.3
attrs                   22.1.0
audioread               3.0.0
backcall                0.2.0
beautifulsoup4          4.11.1
bleach                  5.0.1
blis                    0.7.9
cachetools              5.2.0
catalogue               2.0.8
ccimport                0.4.2
certifi                 2022.9.24
cffi                    1.15.1
charset-normalizer      2.1.1
click                   8.1.3
cloudpickle             2.2.0
cmake                   3.24.1.1
colorama                0.4.6
confection              0.0.3
ConfigArgParse          1.5.3
contourpy               1.0.6
cuda-python             11.7.0+0.g95a2041.dirty
cudf                    22.8.0a0+304.g6ca81bbc78.dirty
cugraph                 22.8.0a0+132.g2daa31b6.dirty
cuml                    22.8.0a0+52.g73b8d00d0.dirty
cumm                    0.4.8
cumm-cu118              0.4.8
cupy-cuda118            11.0.0
cycler                  0.11.0
cymem                   2.0.7
Cython                  0.29.32
dash                    2.9.2
dash-core-components    2.0.0
dash-html-components    2.0.0
dash-table              5.0.0
dask                    2022.7.1
dask-cuda               22.8.0a0+36.g9860cad
dask-cudf               22.8.0a0+304.g6ca81bbc78.dirty
debugpy                 1.6.3
decorator               5.1.1
defusedxml              0.7.1
distributed             2022.7.1
easydict                1.10
entrypoints             0.4
exceptiongroup          1.0.4
execnet                 1.9.0
executing               1.2.0
expecttest              0.1.3
fastjsonschema          2.16.2
fastrlock               0.8.1
fire                    0.5.0
Flask                   2.2.3
fonttools               4.38.0
fsspec                  2022.11.0
functorch               1.13.0a0+936e930
google-auth             2.14.1
google-auth-oauthlib    0.4.6
graphsurgeon            0.4.6
grpcio                  1.50.0
HeapDict                1.0.1
hypothesis              5.35.1
idna                    3.4
imageio                 2.27.0
importlib-metadata      5.0.0
importlib-resources     5.10.0
iniconfig               1.1.1
intel-openmp            2021.4.0
ipykernel               6.17.1
ipython                 8.6.0
ipython-genutils        0.2.0
ipywidgets              8.0.6
itsdangerous            2.1.2
jedi                    0.18.1
Jinja2                  3.1.2
joblib                  1.2.0
json5                   0.9.10
jsonschema              4.17.0
jupyter_client          7.4.7
jupyter_core            5.0.0
jupyter-tensorboard     0.2.0
jupyterlab              2.3.2
jupyterlab-pygments     0.2.2
jupyterlab-server       1.2.0
jupyterlab-widgets      3.0.7
jupytext                1.14.1
kiwisolver              1.4.4
langcodes               3.3.0
lark                    1.1.5
lazy_loader             0.2
librosa                 0.9.2
llvmlite                0.39.1
locket                  1.0.0
Markdown                3.4.1
markdown-it-py          2.2.0
MarkupSafe              2.1.1
matplotlib              3.6.2
matplotlib-inline       0.1.6
mdit-py-plugins         0.3.1
mdurl                   0.1.2
mistune                 2.0.4
mkl                     2021.1.1
mkl-devel               2021.1.1
mkl-include             2021.1.1
mmcv-full               1.5.0
mmdet                   2.22.0                         /workspace/mmdetection-v2.22.0
mock                    4.0.3
model-index             0.1.11
msgpack                 1.0.4
murmurhash              1.0.9
nbclient                0.7.0
nbconvert               7.2.5
nbformat                5.7.0
nest-asyncio            1.5.6
networkx                3.0
ninja                   1.11.1
notebook                6.4.10
numba                   0.56.4
numpy                   1.22.2
nvidia-dali-cuda110     1.18.0
nvidia-pyindex          1.0.9
nvtx                    0.2.5
oauthlib                3.2.2
onnx                    1.12.0
open3d                  0.17.0
opencv                  4.6.0
opencv-python           4.7.0.72
openmim                 0.3.7
ordered-set             4.1.0
packaging               21.3
pandas                  1.4.4
pandocfilters           1.5.0
parso                   0.8.3
partd                   1.3.0
pathy                   0.8.1
pccm                    0.4.6
pcdet                   0.1.0-                         /workspace
pexpect                 4.8.0
pickleshare             0.7.5
Pillow                  9.4.0
pip                     23.0.1
pkgutil_resolve_name    1.3.10
platformdirs            2.5.4
plotly                  5.14.0
pluggy                  1.0.0
polygraphy              0.42.1
pooch                   1.6.0
portalocker             2.7.0
preshed                 3.0.8
prettytable             3.5.0
prometheus-client       0.15.0
prompt-toolkit          3.0.32
protobuf                3.20.1
psutil                  5.9.4
ptyprocess              0.7.0
pure-eval               0.2.2
pyarrow                 8.0.0
pyasn1                  0.4.8
pyasn1-modules          0.2.8
pybind11                2.10.1
pycocotools             2.0.2
pycparser               2.21
pydantic                1.10.2
Pygments                2.13.0
pylibcugraph            22.8.0a0+132.g2daa31b6.dirty
pynvml                  11.4.1
pyparsing               3.0.9
pyquaternion            0.9.9
pyrsistent              0.19.2
pytest                  7.2.0
pytest-rerunfailures    10.2
pytest-shard            0.1.2
pytest-xdist            3.0.2
python-dateutil         2.8.2
python-hostlist         1.22
pytorch-quantization    2.1.2
pytz                    2022.6
PyWavelets              1.4.1
PyYAML                  6.0
pyzmq                   24.0.1
raft                    22.8.0a0+70.g9070c30.dirty
regex                   2022.10.31
requests                2.28.1
requests-oauthlib       1.3.1
resampy                 0.4.2
rich                    13.3.3
rmm                     22.8.0a0+62.gf6bf047.dirty
rsa                     4.9
scikit-image            0.20.0
scikit-learn            0.24.2
scipy                   1.9.1
Send2Trash              1.8.0
setuptools              59.5.0
six                     1.16.0
smart-open              5.2.1
sortedcontainers        2.4.0
soundfile               0.11.0
soupsieve               2.3.2.post1
spacy                   3.4.3
spacy-legacy            3.0.10
spacy-loggers           1.0.3
spconv                  2.3.5
spconv-cu118            2.3.5
sphinx-glpi-theme       0.3
srsly                   2.4.5
stack-data              0.6.1
tabulate                0.9.0
tbb                     2021.7.1
tblib                   1.7.0
tenacity                8.2.2
tensorboard             2.9.0
tensorboard-data-server 0.6.1
tensorboard-plugin-wit  1.8.1
tensorboardX            2.6
tensorrt                8.5.1.7
termcolor               2.2.0
terminado               0.17.0
terminaltables          3.1.10
thinc                   8.1.5
threadpoolctl           3.1.0
tifffile                2023.3.21
tinycss2                1.2.1
toml                    0.10.2
tomli                   2.0.1
toolz                   0.12.0
torch                   1.13.0a0+936e930
torch-tb-profiler       0.4.1
torch-tensorrt          1.3.0a0
torchtext               0.13.0a0+fae8e8c
torchvision             0.15.0a0
tornado                 6.2
tqdm                    4.64.1
traitlets               5.5.0
transformer-engine      0.2.0
treelite                2.4.0
treelite-runtime        2.4.0
typer                   0.7.0
typing_extensions       4.4.0
ucx-py                  0.27.0a0+29.ge9e81f8
uff                     0.6.9
urllib3                 1.26.12
wasabi                  0.10.1
wcwidth                 0.2.5
webencodings            0.5.1
Werkzeug                2.2.3
wheel                   0.38.4
widgetsnbextension      4.0.7
xdoctest                1.0.2
xgboost                 1.6.1
yapf                    0.32.0
zict                    2.2.0
zipp                    3.10.0

We encountered an error with the VoxelGenerator, so we made changes to the code as shown below [issue #10].

The reproducible performance results are as follows.		Easy	Moderate	Hard
Car AP_R40	82.3769	64.4927	58.1761
Pedestrian AP_R40	29.7712	23.7108	20.2043
Cyclist AP_R40	59.8114	34.4488	31.7488

[bringeyes]

@jouhy Even after changing the "VoxelGenerator" to "PointToVoxel", I still encountered the following issue:

File "/data/DSGN2/pcdet/datasets/processor/data_processor.py", line 198, in transform_points_to_voxels voxel_output = voxel_generator.generate(points) AttributeError: 'PointToVoxel' object has no attribute 'generate'

May I know how you resolved this issue when using spconv2.x?

[jouhy]

@bringeyes ./pcdet/models/backbones_3d_lidar/spconv_backbone.py I changed the code in the above file to the following.

import spconv.pytorch as spconv  # instead import spconv

[bringeyes]

@jouhy Yes, I had made the modification you mentioned before encountering the error. The reason for this error is that the original project used spconv1.x version and VoxelGenerator.generate() used in the file ./pcdet/datasets/processor/data_processor.py also comes from spconv1.x. Therefore, merely modifying "VoxelGenerator" to "PointToVoxel" is not sufficient, and subsequent code needs to be modified accordingly. Did you make the corresponding modifications?

[jouhy]

@bringeyes yes, I fixed it in the file ./pcdet/datasets/processor/data_processor.py with:

points = data_dict['points']
points_torch = torch.from_numpy(points)
voxels, coordinates, num_points = voxel_generator(points_torch, empty_mean=True)
# if isinstance(voxel_output, dict):
#     voxels, coordinates, num_points = \
#         voxel_output['voxels'], voxel_output['coordinates'], voxel_output['num_points_per_voxel']
# else:
#     voxels, coordinates, num_points = voxel_output

[bringeyes]

@jouhy Thank you very much for your help! The error has been resolved now!!