search

tukilabs / Video-Compression-Net

A new approach to video compression by refining the shortcomings of conventional approach and substituting each traditional component with their neural network counterpart. Our proposed work consists of motion estimation, compression and compensation and residue compression, learned end-to-end to minimize the rate-distortion trade off. The whole model is jointly optimized using a single loss function. Our work is based on a standard method to exploit the spatio-temporal redundancy in video frames to reduce the bit rate along with the minimization of distortions in decoded frames. We implement a neural network version of conventional video compression approach and encode the redundant frames with lower number of bit.
24 stars 12 forks source link

Motion compensation #1

Open RenYang-home opened 3 years ago

RenYang-home commented 3 years ago

Dear authors,

It is a great efficient network of deep video compression. It seems that there is no motion compensation network and it only uses warping to compensate motion. Will this lead to a drop in performance? Could you please provide some experimental results of this code?

Thanks, Ren

prasangadhungel commented 3 years ago

Dear Ren,

Yes, the doubt that the removal of motion compensation net will lead to drop in performance, is perfectly reasonable. Some experiments performed on Vimeo-90k Dataset (separated as test sequence) have shown similar result or a slight drop in performance compared to DVC but with much gain in terms of speed and memory usage. We have to validate the results in other standard test datasets of different resolutions. As it turned out, general test sequences are not publicly available, we have contacted the authors for the datasets. As soon as we get the datasets and carry out the evaluation we will provide the result in this repo itself.

Thanks, Prasanga Dhungel

RenYang-home commented 3 years ago

What test sequences are you looking for? JCT-VC dataset?

prasangadhungel commented 3 years ago

Dear Ren,

We are thankful for you help and support. Please refer the experiment for the experimental results. As suspected our PSNR optimized model suffers a drop in performance but MS-SSIM optimized models seem to stand the ground.

Regards, Prasanga Dhungel

RenYang-home commented 3 years ago

Dear Prasanga,Thanks for informing me your updates. Please note that the MS-SSIM curves in the original DVC paper are the results of the PSNR trained models. DVC did not train MS-SSIM optimized model in their CVPR paper, but did it in their PAMI paper, refer tohttps://ieeexplore.ieee.org/abstract/document/9072487Best,Ren-------- 原始邮件 --------发件人： PrasangaDhungel notifications@github.com日期： 2020年10月9日周五 15:40收件人： tukilabs/Video-Compression-Net Video-Compression-Net@noreply.github.com抄送： Ren Yang r.yangchn@gmail.com, Author author@noreply.github.com主 题： Re: [tukilabs/Video-Compression-Net] Motion compensation (#1) Dear Ren, We are thankful for you help and support. Please refer the experiment for the experimental results. As suspected our PSNR optimized model suffers a drop in performance but MS-SSIM optimized models seem to stand the ground. Regards, Prasanga Dhungel

—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe.

prasangadhungel commented 3 years ago

Dear Ren,

Thanks for the information, we will look into this further.

Regards, Prasanga Dhungel