search

erdos-project / pylot

Modular autonomous driving platform running on the CARLA simulator and real-world vehicles.
https://pylot.readthedocs.io/
Apache License 2.0
459 stars 132 forks source link

Deep SORT tracking errors on multi-GPU machines #216

Open objorkman opened 2 years ago

objorkman commented 2 years ago

Screenshot: image

To reproduce, run pylot with --flagfile=configs/tracking.conf and tracker_type=deep_sort set in the config file.

pschafhalter commented 2 years ago

Looks like the bug might be because deep sort sends variables to GPU devices in different sections of the code.

This might require a change to the deep sort repo. Looks like self.gaussian_mask is moved to a device in __init__ whereas processed_crops is moved to a device every time run_deep_sort is called (see here). A fix might be to move processed_crops to the same device as self.gaussian_mask -- I think we can do this by passing the right device when calling .cuda().