Batchs

[MazenHamdi]

Hello, great work, I was just trying to follow up the inputs from the batchs to the output to more understand how the model works, but the data in the batches are not very clear,

1/Can you please explain it to me ? 2/Also, I understand there are 4 transformers, 2 spatials and 2 temporals, but the semantic job of each transformer was not very clear, (why are the first 2 transformers parallel and then sequential, and what is the semantic process that they do) 3/what does the find_trajectory_fragment do exactly ? Thanks a lot

[cunjunyu]

1) they are corresponding to

1. Coordinates of pedestrians relative to the last observation frame
2. Adjacent matrix indicating neighbors
3. Number of neighbors of the pedestrian
4. Number of pedestrians in each scene which is used to form the batch

2) The first two: to capture temporal dependency and social interaction individually The second two: to capture temporal dependency combining social information and vice versa

3) To generate a training sample given a starting timestamp in the scence.

[MazenHamdi]

I understand there are 7 components for the batch ,, batch_data = batch, batch - shift_value, shift_value, seq_list, nei_list, nei_num, batch_pednum, correct me if I am wrong: -seq_list: is a 0,1 matrix to determine missing data -nei_list: is a (N,N) shaped array, N number of total pedestrians in the 20 frames: determines the number of neighbors for each pedestrians (if the distance is less than a certain thred they are considered neighbors) -nei_number: the sum of the neighbors for each pedestrian -batch_pednum: total number of pedestrian in each frame

what is not clear is the batch , batch - shift_value, shift_value did you apply data augmentation in the batch_rotate_shift function? If yes, why did you return the batch - shift_value and shift_value . Thanks a lot for your help, I really appreciate it

[cunjunyu]

Coordinates of pedestrians relative to the last observation frame

shift_value is the shift between the current frame and the last observation frame