[Bug] Recent change in local_visualizer.py causes error on Windows

MatthiasHinz commented 10 months ago

Prerequisite

[X] I have searched Issues and Discussions but cannot get the expected help.
[X] I have read the FAQ documentation but cannot get the expected help.
[X] The bug has not been fixed in the latest version (dev-1.x) or latest version (dev-1.0).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmdetection3d

Environment

sys.platform: win32 Python: 3.9.16 | packaged by conda-forge | (main, Feb 1 2023, 21:28:38) [MSC v.1929 64 bit (AMD64)] CUDA available: True numpy_random_seed: 2147483648 GPU 0: NVIDIA GeForce RTX 3080 Ti CUDA_HOME: C:\ProgramData\Anaconda3\envs\mmdetection NVCC: Cuda compilation tools, release 11.6, V11.6.124 GCC: n/a PyTorch: 1.12.0 PyTorch compiling details: PyTorch built with:

C++ Version: 199711
MSVC 192829337
Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
OpenMP 2019
LAPACK is enabled (usually provided by MKL)
CPU capability usage: AVX2
CUDA Runtime 11.6
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
CuDNN 8.3.2 (built against CUDA 11.5)
Magma 2.5.4
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.6, CUDNN_VERSION=8.3.2, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -IC:/cb/pytorch_1000000000000/work/mkl/include -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.13.0 OpenCV: 4.7.0 MMEngine: 0.8.4 MMDetection: 3.0.0 MMDetection3D: 1.3.0+b84ba5a spconv2.0: True

Reproduces the problem - code sample

#

Reproduces the problem - command or script

python mmdetection3d/tools/misc/browsedataset.py .\configs_base\datasets\kitti-3d-3class.py --task lidar_det --output-dir .\data\kitti\ --show-interval 30

Reproduces the problem - error message

Traceback (most recent call last): File "E:\mmdetection3d\tools\misc\browse_dataset.py", line 152, in main() File "E:\mmdetection3d\tools\misc\browse_dataset.py", line 138, in main visualizer.add_datasample( File "C:\ProgramData\Anaconda3\envs\mmstone\lib\site-packages\mmengine\dist\utils.py", line 401, in wrapper return func(*args, *kwargs) File "E:\mmdetection3d\mmdet3d\visualization\local_visualizer.py", line 1082, in add_datasample self.show( File "C:\ProgramData\Anaconda3\envs\mmstone\lib\site-packages\mmengine\dist\utils.py", line 401, in wrapper return func(args, **kwargs) File "E:\mmdetection3d\mmdet3d\visualization\local_visualizer.py", line 868, in show self.view_control.convert_to_pinhole_camera_parameters() # noqa: E501 AttributeError: 'Det3DLocalVisualizer' object has no attribute 'view_control'

Additional information

Hello,

a recent change in the file local_visualizer.py requires that the environment variable DISPLAY exists in order to initialize the Open3d visualizer backend. Since Windows desktops do usually not set a DISPLAY variable, calls to the Det3DLocalVisualizer e.g. with the browse_dataset script, will terminate with the fatal exception shown above.

See: https://github.com/open-mmlab/mmdetection3d/blame/5c0613be29bd2e51771ec5e046d89ba3089887c7/mmdet3d/visualization/local_visualizer.py#L180

The browse_dataset will work normally again after either removing the if-condition or setting up a DISPLAY variable with any value. However, I think it would be good to rework the code section for system compatibility.

giangdip2410 commented 10 months ago

I have the same bug in Window platform. Do you know how to fix it? Thank you,

sunjiahao1999 commented 10 months ago

Thanks for your reminder, we'll re-test on windows and fix this!

dtz6 commented 10 months ago

I got the same bug on Windows platform.

dtz6 commented 9 months ago

Now I got the same bug on Linux: Traceback (most recent call last): File "projects/BEVFusion/demo/multi_modality_demo.py", line 78, in main(args) File "projects/BEVFusion/demo/multi_modality_demo.py", line 64, in main visualizer.add_datasample( File "/data/dengtianze/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/dist/utils.py", line 419, in wrapper return func(*args, *kwargs) File "/data/dengtianze/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet3d/visualization/local_visualizer.py", line 1082, in add_datasample self.show( File "/data/dengtianze/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/dist/utils.py", line 419, in wrapper return func(args, **kwargs) File "/data/dengtianze/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet3d/visualization/local_visualizer.py", line 868, in show self.view_control.convert_to_pinhole_camera_parameters() # noqa: E501 AttributeError: 'Det3DLocalVisualizer' object has no attribute 'view_control'

lypan1208 commented 9 months ago

I've got the same bug, does anyone know how to fix this? Thanks!

MatthiasHinz commented 9 months ago

I've got the same bug, does anyone know how to fix this? Thanks!

Did you try out one of the workarounds that I suggested above?

For now, you could either try setting up an environment variable named DISPLAY or remove the if-clause in the code chunk that I linked. In my case, the error occurred because the following lines 181 and 182 were not executed if this variable is not present.

Be aware that setting up a DISPLAY environment variable might have undesired side effects with other programs that try to read it, so it's really just a workaround.

lypan1208 commented 9 months ago

I've got the same bug, does anyone know how to fix this? Thanks!

Did you try out one of the workarounds that I suggested above?

For now, you could either try setting up an environment variable named DISPLAY or remove the if-clause in the code chunk that I linked. In my case, the error occurred because the following lines 181 and 182 were not executed if this variable is not present.

Be aware that setting up a DISPLAY environment variable might have undesired side effects with other programs that try to read it, so it's really just a workaround.

As you said, remove the if-clause is working! Appreciate for your help, Thank hou!

zyu commented 9 months ago

File "demo/pcd_demo.py", line 90, in main() File "demo/pcd_demo.py", line 80, in main inferencer(call_args) File "e:\mm\mmdetection3d\mmdet3d\apis\inferencers\base_3d_inferencer.py", line 213, in call visualization = self.visualize(ori_inputs, preds, File "e:\mm\mmdetection3d\mmdet3d\apis\inferencers\lidar_det3d_inferencer.py", line 190, in visualize self.visualizer.add_datasample( File "C:\ProgramData\miniconda3\envs\openmmlab\lib\site-packages\mmengine\dist\utils.py", line 418, in wrapper return func(*args, *kwargs) File "e:\mm\mmdetection3d\mmdet3d\visualization\local_visualizer.py", line 1082, in add_datasample self.show( File "C:\ProgramData\miniconda3\envs\openmmlab\lib\site-packages\mmengine\dist\utils.py", line 418, in wrapper return func(args, kwargs) File "e:\mm\mmdetection3d\mmdet3d\visualization\local_visualizer.py", line 868, in show self.view_control.convert_to_pinhole_camera_parameters() # noqa: E501 AttributeError: 'Det3DLocalVisualizer' object has no attribute 'view_control'

z-mahmud22 commented 8 months ago

Thank you @MatthiasHinz for mentioning the issue and suggesting a workaround.

I followed your suggestion by commenting out line 180. However, I was having this error:

TypeError: paint_uniform_color(): incompatible function arguments. The following argument types are supported:
    1. (self: open3d.cuda.pybind.geometry.LineSet, color: numpy.ndarray[numpy.float64[3, 1]]) -> open3d.cuda.pybind.geometry.LineSet

Invoked with: LineSet with 12 lines., 0.0

Additionally, I had to make some changes to the local_visualizer.py file following this PR #2729 to get rid of that error. Now I can visualize the point cloud with predicted 3D bounding boxes. This is my testing environment:

OS: Windows 11 GPU: NVIDIA GeForce RTX 4090 CUDA: 11.8 Python: 3.8 Pytorch: 2.1.2 TorchVision: 0.16.2 OpenCV: 4.9.0.80 Open3D: 0.18.0 MMEngine: 0.10.2 MMDetection: 3.3.0 MMDetection3D: 1.4.0 (Built from source)

GODBENGAY commented 8 months ago

Thank you @MatthiasHinz for mentioning the issue and suggesting a workaround.

I followed your suggestion by commenting out line 180. However, I was having this error:
TypeError: paint_uniform_color(): incompatible function arguments. The following argument types are supported:
    1. (self: open3d.cuda.pybind.geometry.LineSet, color: numpy.ndarray[numpy.float64[3, 1]]) -> open3d.cuda.pybind.geometry.LineSet

Invoked with: LineSet with 12 lines., 0.0
Additionally, I had to make some changes to the file following this PR #2729 to get rid of that error. Now I can visualize the point cloud with predicted 3D bounding boxes. This is my testing environment:local_visualizer.py

OS: Windows 11 GPU: NVIDIA GeForce RTX 4090 CUDA: 11.8 Python: 3.8 Pytorch: 2.1.2 TorchVision: 0.16.2 OpenCV: 4.9.0.80 Open3D: 0.18.0 MMEngine: 0.10.2 MMDetection: 3.3.0 MMDetection3D: 1.4.0 (Built from source)

Can I See your local_visualizer.py？ I still can't run it

hitbuyi commented 6 months ago

I got the same bug on Windows platform.

hitbuyi commented 6 months ago

Thanks for your reminder, we'll re-test on windows and fix this!

this bug still exists in mmdet3D v1.4.0 which is released recently

zkailinzhang commented 2 months ago

same bug mmcv 2.1.0 mmdet 3.3.0 mmdet3d 1.4.0 mmengine 0.10.4

zkailinzhang commented 2 months ago

import os os.environ['DISPLAY'] = '1',,,可以解决

open-mmlab / mmdetection3d