qiuyu96 / CoDeF

[CVPR 2024 Highlight] Official PyTorch implementation of CoDeF: Content Deformation Fields for Temporally Consistent Video Processing
https://qiuyu96.github.io/CoDeF/
Other
4.83k stars 388 forks source link

about run app.py (Segment-and-track-anything) in long video(images>100),result have error #39

Open zhanghongyong123456 opened 1 year ago

zhanghongyong123456 commented 1 year ago

image image

zhanghongyong123456 commented 1 year ago

when i set sam_gap > my video img num,and Reset button update,it's OK,but output masks Several more masks (new.png),need remove new.png

qiuyu96 commented 1 year ago

How about setting minimum = 1 here ?

zhanghongyong123456 commented 1 year ago

How about setting minimum = 1 here ?

The same result, can be tracked correctly, but the masks are still a few more masks (new.png),need remove new.png

image

LiNO3Dy commented 1 year ago

The model automatically detects and adds new objects every sam_gap frames. To solve this issue, you can adjust the default value of sam_gap in SegTrackerArgs to its maximum value and click reset.

zhanghongyong123456 commented 1 year ago

The model automatically detects and adds new objects every sam_gap frames. To solve this issue, you can adjust the default value of sam_gap in SegTrackerArgs to its maximum value and click reset.

I test set sam_gap = 9999, The same result, can be tracked correctly, but the masks are still a few more masks (*new.png), image

zhanghongyong123456 commented 1 year ago

How about setting minimum = 1 here ?

The results of my training are not very good, is it because I exercise too fast?, my video fps=30, base.yaml fps=15, need set 5 or 1 it is OK? image image image

Setting those hyperparameters can improve the reconstruction effect of the model

ken-ouyang commented 1 year ago

@zhanghongyong123456 Yes. The motion cannot be so fast as (x,y,t) is viewed as a 3D volume.

zhanghongyong123456 commented 1 year ago

May I ask if you will publish a document about training hyperparameter settings. I found that many people are asking about long videos and fast exercise training skills. I saw your reply, but we don’t know how to set them up.

image https://github.com/qiuyu96/CoDeF/issues/2 image https://github.com/qiuyu96/CoDeF/issues/20#issuecomment-1686238327 image https://github.com/qiuyu96/CoDeF/issues/31#issuecomment-1689242014 image https://github.com/qiuyu96/CoDeF/issues/36#issuecomment-1689282934 https://github.com/qiuyu96/CoDeF/issues/41#issuecomment-1691005920