RuntimeError: python value of type 'QuaternionCoeffOrder' cannot be used as a value

[BBBeatrice]

Hi, @BroJunn thanks for your great work! I followed the instruction in ReadMe and tried to test on Argo2 dataset. However, I meet this error when I run this line"python sensor_fusion/main.py --root_path /my_argo2_path", more info of the error as below.

Traceback (most recent call last):
  File "sensor_fusion/main.py", line 12, in <module>
    from utils_sf.scene_dataset import SceneDataset
  File "/home/grads/yuxindu8/sensor_fusion_of_lidar_cams/sensor_fusion/utils_sf/scene_dataset.py", line 1, in <module>
    from pcdet.datasets import DatasetTemplate
  File "/home/grads/yuxindu8/OpenPCDet/pcdet/datasets/__init__.py", line 15, in <module>
    from .argo2.argo2_dataset import Argo2Dataset
  File "/home/grads/yuxindu8/OpenPCDet/pcdet/datasets/argo2/argo2_dataset.py", line 15, in <module>
    from .argo2_utils.so3 import yaw_to_quat, quat_to_yaw
  File "/home/grads/yuxindu8/OpenPCDet/pcdet/datasets/argo2/argo2_utils/so3.py", line 10, in <module>
    def quat_to_mat(quat_wxyz: Tensor) -> Tensor:
  File "/home/grads/yuxindu8/anaconda3/envs/py38/lib/python3.8/site-packages/torch/jit/__init__.py", line 1550, in script
    fn = torch._C._jit_script_compile(qualified_name, ast, _rcb, get_default_args(obj))
  File "/home/grads/yuxindu8/anaconda3/envs/py38/lib/python3.8/site-packages/torch/jit/_recursive.py", line 583, in try_compile_fn
    return torch.jit.script(fn, _rcb=rcb)
  File "/home/grads/yuxindu8/anaconda3/envs/py38/lib/python3.8/site-packages/torch/jit/__init__.py", line 1550, in script
    fn = torch._C._jit_script_compile(qualified_name, ast, _rcb, get_default_args(obj))
RuntimeError: 
python value of type 'QuaternionCoeffOrder' cannot be used as a value:
  File "/home/grads/yuxindu8/anaconda3/envs/py38/lib/python3.8/site-packages/kornia/geometry/conversions.py", line 492
            raise ValueError(f"order must be one of {QuaternionCoeffOrder.__members__.keys()}")

    if order == QuaternionCoeffOrder.XYZW:
                ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
        warnings.warn(
            "`XYZW` quaternion coefficient order is deprecated and"
'quaternion_to_rotation_matrix' is being compiled since it was called from 'quat_to_mat'
  File "/home/grads/yuxindu8/OpenPCDet/pcdet/datasets/argo2/argo2_utils/so3.py", line 19
        (...,3,3) 3D rotation matrices.
    """
    return C.quaternion_to_rotation_matrix(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~...  <--- HERE
        quat_wxyz, order=C.QuaternionCoeffOrder.XYZW
    )

I am using a cluster with cuda10.1 pytorch1.6.0 spconv2.3.6 gcc7.5.0

[BroJunn]

Hi @BBBeatrice, it seems the problem is from the version of either torch or kornia. According to the error message, there should be strong dependencies of these two packages, I guess. My pytorch should be 1.10.0 and kornia is 0.5.8, I'll check it later. So my advice is to either use newer version pytorch or degrade kornia.

[BBBeatrice]

Thank you for your reply! The error is definitely about the mismatch version of different parts. I changed different version and it has worked well now! For more detail, my current torch version is '1.8.1+cu101', my kornia is 0.5.8.

By the way, since I am new to this field, to my knowledge, the output result is the 3D detection results by fusion of camera and lidar input. Is my thought right? Also, can I get evaluation results like average precision or IOU(kitti evaluation metrics) by using the output detection results and label ground truth? Thank you again for your brilliant work!

[BroJunn]

You're right, I just use two filtering (confidence threshold and 2d detection) to filter the nasty 3d detection. There're some mistakes I made (e.g. dt in KF should be 0.1) and I will release a fixed version also with a nicer visualization in a few days. Unfortunately, I'm also new to this filed. This is just a starting task to get known with the basic stuffs since I would like to pursue a phd degree in this field. Regarding the metrics you said, of course, you can use the 3d detection results for cls and detect evaluation (I think). The whole repo is more like a practical test for so-called post-fusion, not a very powerful fusion pipeline from my perspective. ; ) There should be still points for imporvement.

[BBBeatrice]

You're right, I just use two filtering (confidence threshold and 2d detection) to filter the nasty 3d detection. There're some mistakes I made (e.g. dt in KF should be 0.1) and I will release a fixed version also with a nicer visualization in a few days. Unfortunately, I'm also new to this filed. This is just a starting task to get known with the basic stuffs since I would like to pursue a phd degree in this field. Regarding the metrics you said, of course, you can use the 3d detection results for cls and detect evaluation (I think). The whole repo is more like a practical test for so-called post-fusion, not a very powerful fusion pipeline from my perspective. ; ) There should be still points for imporvement.

So many thanks,bro! Since I am also doing researches based on the dataset argoverse, your work helps a lot! Looking forward to your fixed version!