Doubt on the architecture

[ambarish1999]

Are we using a different LSTM module per track ? If we directly feed all object Xt and the corresponding flow feature, how is the track information maintained?

[MoonBlvd]

Hi @ambarish1999 , we use the same GRU for all objects. But each object's feature run through the network separately - you can batch all objects as a minibatch to send into the network for sure.

[ambarish1999]

Few more queries. 1--> Did you retrain deep sort for vehicles , as it tends to give wrong track information frequently on the pre trained weights?

2-->Secondly, the pickle file is marked by video_id_object_id, are each of these files holding information of the entire track for that object? If that's so, during testing , is it possible to do online detection, where the entire track information isnt there but the frames will come on the fly from the video. My doubt is during testing is this model expecting pre processing of the full video to construct all track informations and then feed it, or we can feed frame by frame to it?

[MoonBlvd]

1. No I did not. My visual observation was that the tracking was not too bad with the provided weights.
2. Great quetsion. We can feed frame by frame since it's online RNN style. Unfortunately we did not provide a multi-car inferencing code since we did not necessarily need it for our experiment. However it should be fairly easy to write one since you can batch multiple cars and send to the network to predict one trajectory for each fo them.

For the tracking - you don't need to run tracking for the whole viedeo and then run the fol model. What you can do is to run a loop with ...detction(t) -> track(t) -> anomly detection(t) -> detection (t+1) -> track (t+1) -> anomaly detection (t+1) ->..., does it make sense?