search

microsoft / DCVC

Deep Contextual Video Compression
MIT License
342 stars 54 forks source link

performance drop when fintunning DMC model from ACMMM2022 #2

Open jinxixiang opened 1 year ago

jinxixiang commented 1 year ago

Congratulations on such remarkable results that surpass VVC!

I ran the test code with the pre-trained model you provided and I reproduced the results you reported in the paper.

Unluckily, when I tried to finetune it for some steps following the training strategy (lr=1e-5, multi-frame forward loss averaging) in (sheng et al2021), the RD curve will drop around 0.7 dB after just hundreds of steps.

The training code is based on DVC and the regular rate-distortion loss is optimized.

Any suggestions will be appreciated.

Thank you!

kesixcross commented 1 year ago

I ran into the same problem. Performance drops just a few steps after fine-tuning. I wonder if the authors can release their train code. Any suggestions will be appreciated. Thank you!

mosquitobite commented 1 year ago

Hi, do you solve the problem?

kesixcross commented 1 year ago

Hi, do you solve the problem?

sorry, I have not solved the problem.

tl32rodan commented 1 year ago

Same here. It's just fine to fine-tune DCVC-TCM, but fail on DCVC-HEM and DCVC-DC cases. I guess training "adaptive quantization" could be non-trivial, but there's no quite much training details mentioned in the papers. It'll be a lot helpful if the author by any chance can share more training details, thanks.

guohf3 commented 1 year ago

Me too. It would be appreciated if anyone solved this problem and shared it.

MingZwhy commented 1 year ago

Congratulations on such remarkable results that surpass VVC!

I ran the test code with the pre-trained model you provided and I reproduced the results you reported in the paper.

Unluckily, when I tried to finetune it for some steps following the training strategy (lr=1e-5, multi-frame forward loss averaging) in (sheng et al2021), the RD curve will drop around 0.7 dB after just hundreds of steps.

The training code is based on DVC and the regular rate-distortion loss is optimized.

Any suggestions will be appreciated.

Thank you!

Hi！ I also want to fine-tune DCVC for my research, but I don't know how to train it. Could you share your training code with me? I will appreciate it if you can help me on fine-tune DCVC model !

wx-Tian commented 7 months ago

Same here. It's just fine to fine-tune DCVC-TCM, but fail on DCVC-HEM and DCVC-DC cases. I guess training "adaptive quantization" could be non-trivial, but there's no quite much training details mentioned in the papers. It'll be a lot helpful if the author by any chance can share more training details, thanks.

@tl32rodan I use the training code of the compressai to fine-tune DCVC-TCM for 1 epoch, the performance drops highly, I don't know why cause this, could you share your fine-tune code for me?

tl32rodan commented 7 months ago

Hi Tian, I’m actually no longer working on this topic now, but one major thing you might need to be careful about is the quantization. You should use additive noise for quantization instead of rounding, just like their released code does, during training. If you already done this, I’m afraid I may not be able to provide further details. Wish you the best.

BR

wenxin Tian @.***>於 2023年11月23日 週四，下午11:46寫道：

Same here. It's just fine to fine-tune DCVC-TCM, but fail on DCVC-HEM and DCVC-DC cases. I guess training "adaptive quantization" could be non-trivial, but there's no quite much training details mentioned in the papers. It'll be a lot helpful if the author by any chance can share more training details, thanks.

@tl32rodan https://github.com/tl32rodan I use the training code of the compressai to fine-tune DCVC-TCM for 1 epoch, the performance drops highly, I don't know why cause this, could you share your fine-tune code for me?

— Reply to this email directly, view it on GitHub https://github.com/microsoft/DCVC/issues/2#issuecomment-1824241040, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJNI6RZPJVLW3AEC76SARBDYF4VN7AVCNFSM55UTXZJKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBSGQZDIMJQGQYA . You are receiving this because you were mentioned.Message ID: @.***>

wx-Tian commented 7 months ago

Hi Tian, I’m actually no longer working on this topic now, but one major thing you might need to be careful about is the quantization. You should use additive noise for quantization instead of rounding, just like their released code does, during training. If you already done this, I’m afraid I may not be able to provide further details. Wish you the best. BR wenxin Tian @.>於 2023年11月23日 週四，下午11:46寫道： … Same here. It's just fine to fine-tune DCVC-TCM, but fail on DCVC-HEM and DCVC-DC cases. I guess training "adaptive quantization" could be non-trivial, but there's no quite much training details mentioned in the papers. It'll be a lot helpful if the author by any chance can share more training details, thanks. @tl32rodan https://github.com/tl32rodan I use the training code of the compressai to fine-tune DCVC-TCM for 1 epoch, the performance drops highly, I don't know why cause this, could you share your fine-tune code for me? — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJNI6RZPJVLW3AEC76SARBDYF4VN7AVCNFSM55UTXZJKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBSGQZDIMJQGQYA . You are receiving this because you were mentioned.Message ID: @.>

Thanks for your reply!!!

YY-RR-ZZ commented 2 months ago

Same here. It's just fine to fine-tune DCVC-TCM, but fail on DCVC-HEM and DCVC-DC cases. I guess training "adaptive quantization" could be non-trivial, but there's no quite much training details mentioned in the papers. It'll be a lot helpful if the author by any chance can share more training details, thanks.

Hello, I'm very interested in DCVC-TCM, but I don't have the ability to fully reproduce the training phase of the code. I'm wondering if you would be willing to share the training phase code with me. My email is 1059561754@qq.com. I would greatly appreciate it!