MMCV Compatibility Issues When Setting Up The Environment on AGX Orin

[HasarinduPerera]

Hi there, I am trying to run UniAD on AGX Orin platform.

uname -a Linux nvidia-desktop 5.10.120-tegra #4 SMP PREEMPT aarch64 aarch64 aarch64 GNU/Linux

nvcc -v

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Sun_Oct_23_22:16:07_PDT_2022
Cuda compilation tools, release 11.4, V11.4.315
Build cuda_11.4.r11.4/compiler.31964100_0

I am getting below error when I try to run ./tools/uniad_dist_eval.sh ./projects/configs/stage2_e2e/base_e2e.py /home/nvidia/UniAD/ckpts/uniad_base_e2e.pth 1

Traceback (most recent call last):
  File "./tools/test.py", line 10, in <module>
    from mmcv.parallel import MMDataParallel
ModuleNotFoundError: No module named 'mmcv.parallel'
/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See 
https://pytorch.org/docs/stable/distributed.html#launch-utility for 
further instructions

  warnings.warn(
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 3427) of binary: /usr/bin/python
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 715, in run
    elastic_launch(
  File "/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
./tools/test.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-03-11_14:03:51
  host      : nvidia-desktop
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 3427)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================

MMCV is already installed, but mmcv.parallel is not there. I tried multiple versions of MMCV including 2.0.0rc4, 2.0.0rc3, 1.7.2

Python 3.8.10 (default, Nov 22 2023, 10:22:35) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from mmcv.parallel import MMDataParallel
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'mmcv.parallel'
>>> import mmcv
>>> mmcv.__version__
'2.0.0'
>>>