Issue with bird's eye view reconstruction

Dear Dr. Dendorfer,

Firstly, I greatly admire your work on MOT and I am very inspired by it!

I am trying to run this code on my custom dataset. As a first step, I am attempting to do the bird's eye view reconstruction. I have been able to run run_depth_estimator.py and run_panoptic_segmentation.py, however, I am getting an error when I am trying to run_homography.py.

Initially, I thought it was an issue with my dataset since I am working on animals instead of pedestrians. However, I encountered the same error when trying to run run_homography.py on the MOT datasets as well.

Below is a rundown of what I did for MOT17-09

I ran run_depth_estimator.py

python src/run_depth_estimator.py --model_path ./data/AdaBins_weights/AdaBins_synthetic.pt --input_path ./data/MOT17/sequences/MOT17-09/img1 --output_path ./data/MOT17/sequences/MOT17-09/depth --images 000001.jpg

I ran run_panoptic_segmentation.py

python src/run_panoptic_segmentation.py --dataset MOT17 --sequences MOT17-09

This created a new directory under the data/MOT17/sequences directory, and I moved the dets, segmentation, and panoptic directories and its contents under the data/MOT17/sequences/MOT17-09 directory

I then tried to run run_homography.py

python src/run_homography.py --dataset MOT17 --sequence MOT17-09

This then resulted in the following error message

Traceback (most recent call last):
  File "/QuoVadis/src/run_homography.py", line 11, in <module>
    run_homography(args.dataset, args.sequence)
  File "/QuoVadis/src/quovadis/bev_reconstruction/homography.py", line 69, in run_homography
    item = seq.__getitem__(frame, ["panoptic"])
  File "/QuoVadis/src/quovadis/datasets/dataset.py", line 1466, in __getitem__
    return {field: callback(self, key) for field, callback in self.frame_cb.items() if field in fields}
  File "/QuoVadis/src/quovadis/datasets/dataset.py", line 1466, in <dictcomp>
    return {field: callback(self, key) for field, callback in self.frame_cb.items() if field in fields}
  File "/QuoVadis/src/quovadis/datasets/utils.py", line 531, in _panoptic_frame_cb
    class_img, mask = panoptic_loader.get_panoptic_img_MOTSynth(panoptic_png)
AttributeError: 'PanopticLoader' object has no attribute 'get_panoptic_img_MOTSynth'

I attempted a few things to address this issue, but I kept running into different attribute errors or value errors. I would be immensely grateful if you could kindly guide me in solving this issue. It would really help me in my research efforts.

I would also like to ask for clarification about the data structure. I noticed that sequences had both the gt and the det. I understand how to format my custom dataset for ground truth. I understand it is the standard in the MOT Benchmark

frame, id, bb_left, bb_top, bb_width, bb_height, conf, x, y, z

I am unsure how to create the det. Is det just the detections without the ID?

Is it as follows? Since there is only one class, the class is just -1?

frame, class, bb_left, bb_top, bb_width, bb_height, conf

Do I need both gt and det to make the code run properly?

Thank you again for your incredible work! Looking forward to hearing from you.

Best, Mae.

dendorferpatrick / QuoVadis

Issue with bird's eye view reconstruction #4