[Bug] Recent change in local_visualizer.py causes error on Windows

MatthiasHinz commented 1 year ago

Prerequisite

[X] I have searched Issues and Discussions but cannot get the expected help.
[X] I have read the FAQ documentation but cannot get the expected help.
[X] The bug has not been fixed in the latest version (dev-1.x) or latest version (dev-1.0).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmdetection3d

Environment

sys.platform: win32 Python: 3.9.16 | packaged by conda-forge | (main, Feb 1 2023, 21:28:38) [MSC v.1929 64 bit (AMD64)] CUDA available: True numpy_random_seed: 2147483648 GPU 0: NVIDIA GeForce RTX 3080 Ti CUDA_HOME: C:\ProgramData\Anaconda3\envs\mmdetection NVCC: Cuda compilation tools, release 11.6, V11.6.124 GCC: n/a PyTorch: 1.12.0 PyTorch compiling details: PyTorch built with:

C++ Version: 199711
MSVC 192829337
Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
OpenMP 2019
LAPACK is enabled (usually provided by MKL)
CPU capability usage: AVX2
CUDA Runtime 11.6
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
CuDNN 8.3.2 (built against CUDA 11.5)
Magma 2.5.4
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.6, CUDNN_VERSION=8.3.2, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -IC:/cb/pytorch_1000000000000/work/mkl/include -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.13.0 OpenCV: 4.7.0 MMEngine: 0.8.4 MMDetection: 3.0.0 MMDetection3D: 1.3.0+b84ba5a spconv2.0: True

Reproduces the problem - code sample

#

Reproduces the problem - command or script

python mmdetection3d/tools/misc/browsedataset.py .\configs_base\datasets\kitti-3d-3class.py --task lidar_det --output-dir .\data\kitti\ --show-interval 30

Reproduces the problem - error message

Traceback (most recent call last): File "E:\mmdetection3d\tools\misc\browse_dataset.py", line 152, in main() File "E:\mmdetection3d\tools\misc\browse_dataset.py", line 138, in main visualizer.add_datasample( File "C:\ProgramData\Anaconda3\envs\mmstone\lib\site-packages\mmengine\dist\utils.py", line 401, in wrapper return func(*args, *kwargs) File "E:\mmdetection3d\mmdet3d\visualization\local_visualizer.py", line 1082, in add_datasample self.show( File "C:\ProgramData\Anaconda3\envs\mmstone\lib\site-packages\mmengine\dist\utils.py", line 401, in wrapper return func(args, **kwargs) File "E:\mmdetection3d\mmdet3d\visualization\local_visualizer.py", line 868, in show self.view_control.convert_to_pinhole_camera_parameters() # noqa: E501 AttributeError: 'Det3DLocalVisualizer' object has no attribute 'view_control'

Additional information

Hello,

a recent change in the file local_visualizer.py requires that the environment variable DISPLAY exists in order to initialize the Open3d visualizer backend. Since Windows desktops do usually not set a DISPLAY variable, calls to the Det3DLocalVisualizer e.g. with the browse_dataset script, will terminate with the fatal exception shown above.

See: https://github.com/open-mmlab/mmdetection3d/blame/5c0613be29bd2e51771ec5e046d89ba3089887c7/mmdet3d/visualization/local_visualizer.py#L180

The browse_dataset will work normally again after either removing the if-condition or setting up a DISPLAY variable with any value. However, I think it would be good to rework the code section for system compatibility.

giangdip2410 commented 12 months ago

I have the same bug in Window platform. Do you know how to fix it? Thank you,

sunjiahao1999 commented 12 months ago

Thanks for your reminder, we'll re-test on windows and fix this!

dtz6 commented 12 months ago

I got the same bug on Windows platform.

dtz6 commented 12 months ago

Now I got the same bug on Linux: Traceback (most recent call last): File "projects/BEVFusion/demo/multi_modality_demo.py", line 78, in main(args) File "projects/BEVFusion/demo/multi_modality_demo.py", line 64, in main visualizer.add_datasample( File "/data/dengtianze/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/dist/utils.py", line 419, in wrapper return func(*args, *kwargs) File "/data/dengtianze/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet3d/visualization/local_visualizer.py", line 1082, in add_datasample self.show( File "/data/dengtianze/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/dist/utils.py", line 419, in wrapper return func(args, **kwargs) File "/data/dengtianze/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet3d/visualization/local_visualizer.py", line 868, in show self.view_control.convert_to_pinhole_camera_parameters() # noqa: E501 AttributeError: 'Det3DLocalVisualizer' object has no attribute 'view_control'

lypan1208 commented 11 months ago

I've got the same bug, does anyone know how to fix this? Thanks!

MatthiasHinz commented 11 months ago

I've got the same bug, does anyone know how to fix this? Thanks!

Did you try out one of the workarounds that I suggested above?

For now, you could either try setting up an environment variable named DISPLAY or remove the if-clause in the code chunk that I linked. In my case, the error occurred because the following lines 181 and 182 were not executed if this variable is not present.

Be aware that setting up a DISPLAY environment variable might have undesired side effects with other programs that try to read it, so it's really just a workaround.

lypan1208 commented 11 months ago

I've got the same bug, does anyone know how to fix this? Thanks!

Did you try out one of the workarounds that I suggested above?

For now, you could either try setting up an environment variable named DISPLAY or remove the if-clause in the code chunk that I linked. In my case, the error occurred because the following lines 181 and 182 were not executed if this variable is not present.

Be aware that setting up a DISPLAY environment variable might have undesired side effects with other programs that try to read it, so it's really just a workaround.

As you said, remove the if-clause is working! Appreciate for your help, Thank hou!

zyu commented 11 months ago

File "demo/pcd_demo.py", line 90, in main() File "demo/pcd_demo.py", line 80, in main inferencer(call_args) File "e:\mm\mmdetection3d\mmdet3d\apis\inferencers\base_3d_inferencer.py", line 213, in call visualization = self.visualize(ori_inputs, preds, File "e:\mm\mmdetection3d\mmdet3d\apis\inferencers\lidar_det3d_inferencer.py", line 190, in visualize self.visualizer.add_datasample( File "C:\ProgramData\miniconda3\envs\openmmlab\lib\site-packages\mmengine\dist\utils.py", line 418, in wrapper return func(*args, *kwargs) File "e:\mm\mmdetection3d\mmdet3d\visualization\local_visualizer.py", line 1082, in add_datasample self.show( File "C:\ProgramData\miniconda3\envs\openmmlab\lib\site-packages\mmengine\dist\utils.py", line 418, in wrapper return func(args, kwargs) File "e:\mm\mmdetection3d\mmdet3d\visualization\local_visualizer.py", line 868, in show self.view_control.convert_to_pinhole_camera_parameters() # noqa: E501 AttributeError: 'Det3DLocalVisualizer' object has no attribute 'view_control'

z-mahmud22 commented 10 months ago

Thank you @MatthiasHinz for mentioning the issue and suggesting a workaround.

I followed your suggestion by commenting out line 180. However, I was having this error:

TypeError: paint_uniform_color(): incompatible function arguments. The following argument types are supported:
    1. (self: open3d.cuda.pybind.geometry.LineSet, color: numpy.ndarray[numpy.float64[3, 1]]) -> open3d.cuda.pybind.geometry.LineSet

Invoked with: LineSet with 12 lines., 0.0

Additionally, I had to make some changes to the local_visualizer.py file following this PR #2729 to get rid of that error. Now I can visualize the point cloud with predicted 3D bounding boxes. This is my testing environment:

OS: Windows 11 GPU: NVIDIA GeForce RTX 4090 CUDA: 11.8 Python: 3.8 Pytorch: 2.1.2 TorchVision: 0.16.2 OpenCV: 4.9.0.80 Open3D: 0.18.0 MMEngine: 0.10.2 MMDetection: 3.3.0 MMDetection3D: 1.4.0 (Built from source)

GODBENGAY commented 10 months ago

Thank you @MatthiasHinz for mentioning the issue and suggesting a workaround.

I followed your suggestion by commenting out line 180. However, I was having this error:
TypeError: paint_uniform_color(): incompatible function arguments. The following argument types are supported:
    1. (self: open3d.cuda.pybind.geometry.LineSet, color: numpy.ndarray[numpy.float64[3, 1]]) -> open3d.cuda.pybind.geometry.LineSet

Invoked with: LineSet with 12 lines., 0.0
Additionally, I had to make some changes to the file following this PR #2729 to get rid of that error. Now I can visualize the point cloud with predicted 3D bounding boxes. This is my testing environment:local_visualizer.py

OS: Windows 11 GPU: NVIDIA GeForce RTX 4090 CUDA: 11.8 Python: 3.8 Pytorch: 2.1.2 TorchVision: 0.16.2 OpenCV: 4.9.0.80 Open3D: 0.18.0 MMEngine: 0.10.2 MMDetection: 3.3.0 MMDetection3D: 1.4.0 (Built from source)

Can I See your local_visualizer.py？ I still can't run it

hitbuyi commented 8 months ago

I got the same bug on Windows platform.

hitbuyi commented 8 months ago

Thanks for your reminder, we'll re-test on windows and fix this!

this bug still exists in mmdet3D v1.4.0 which is released recently

zkailinzhang commented 5 months ago

same bug mmcv 2.1.0 mmdet 3.3.0 mmdet3d 1.4.0 mmengine 0.10.4

zkailinzhang commented 5 months ago

import os os.environ['DISPLAY'] = '1',,,可以解决

open-mmlab / mmdetection3d