Open SJTUwxz opened 1 year ago
Maybe you can follow https://github.com/MCG-NJU/DDM/blob/main/GUIDE.md and find out the reason.
I guess the problem may be frame extraction or pickle preparation.
Thanks for quick response! Yeah I followed that guide and ignore videos that I couldn't download from Youtube anymore. I will take a closer look and check again. I have another two questions:
Thanks again!
Hi. I'm also experiencing the same problem with @SJTUwxz regarding reimplementation, so could you (@JackyTown) release the "multi-frames-GEBD-train-5.pkl" and "multi-frames-GEBD-val-5.pkl" you used for troubleshooting?
Have you tried this in generate_pickle.py?
-shift = np.arange(-frame_per_side, frame_per_side) -shift[shift >= 0] += 1 +shift = np.arange(-frame_per_side, frame_per_side + 1)
Hi. I also have trouble with the reimplementation. Same to @SJTUwxz, I got a 2% performance drop on the validation set with both the model trained by myself and the provided pre-trained one. I wonder if you can release the "multi-frames-GEBD-val-5.pkl" so that I could check out the possible reasons during validation.
Thanks!
Hello, I encounter some difficulties in preparing the Kinetics-GEBD data, could you give me some detailed help? And thanks for your excellent work!
Solved, thanks!
Thanks for quick response! Yeah I followed that guide and ignore videos that I couldn't download from Youtube anymore. I will take a closer look and check again. I have another two questions:
- Are the "multi-frames-GEBD-train-5.pkl" file and also val file released? I know that these pickle files are generated in data loading, so I just want to see if I can compare mine with yours and see if that's causing the trouble.
- It seems that in the data loading, the number of frames of the clip centering query frame is only 10 (T=10), 5 frames before the center frame and 5 frames after the center frame, where the center frame is not loaded in the sequence. This is causing inconsistency because the positional embeddings are initialized with num_locations = 11. I've changed this part of the original code to put center frame back to the sequence because in the paper it says 11 frames are loaded for each clip.
Thanks again!
Hello! @SJTUwxz I also encountered the second issue you mentioned, but I haven't figured out how to put the center frame back to the sequence. Can you tell me which folder it is in
你在generate_pickle.py中尝试过这个吗? -shift = np.arange(-frame_per_side,frame_per_side) -shift[shift >= 0] += 1 +shift = np.arange(-frame_per_side,frame_per_side + 1)
I solved this problem. I found that the above modifications were not made in the file DDM-Net/datasets/MultiFDataset.py. my question!
Hi, this project is great and thanks for releasing the code! I've re-trained DMM and the evaluation result on GEBD val set is as follows, which is around 2% lower than the reported result.
+GEBD Performance on Kinetics-GEBD----+--------+--------+--------+--------+--------+--------+--------+--------+ | Rel.Dis. | 0.05 | 0.10 | 0.15 | 0.20 | 0.25 | 0.30 | 0.35 | 0.40 | 0.45 | 0.50 | Avg | +----------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+ | F1 | 0.7447 | 0.8252 | 0.8496 | 0.8615 | 0.8679 | 0.8722 | 0.8750 | 0.8774 | 0.8796 | 0.8817 | 0.8535 | +----------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+
I've also tried loading the trained weights you've released and run the evaluation again, the result is still around 2% lower, which is,
+GEBD Performance on Kinetics-GEBD----+--------+--------+--------+--------+--------+--------+--------+--------+ | Rel.Dis. | 0.05 | 0.10 | 0.15 | 0.20 | 0.25 | 0.30 | 0.35 | 0.40 | 0.45 | 0.50 | Avg | +----------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+ | F1 | 0.7462 | 0.8234 | 0.8462 | 0.8578 | 0.8642 | 0.8684 | 0.8715 | 0.8739 | 0.8758 | 0.8776 | 0.8505 | +----------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+.
I would really appreciate it if you could provide any insights on possible reasons of this. Thanks a lot!