search

XinyiYing / D3Dnet

Repository for "Deformable 3D Convolution for Video Super-Resolution", SPL, 2020
Apache License 2.0
305 stars 43 forks source link

Question about D3D operation. #19

Closed YDDDDG closed 3 years ago

YDDDDG commented 3 years ago

Hi, I replace 3Dconv with your 3D-Deformable-Cov in my model. However, the GPU memory cost increases from about 4000M to more than 10,000M. Even I adjusted the batch size from 8 to 1 and the path size from128128 to 1616, it still OUT OF MEMORY.

I wanna know whether the D3D operation does require very much GPU memory or maybe there is something wrong with my model. Hope for your reply!

XinyiYing commented 3 years ago

D3Dnet cascaded 5 residual D3Ds (i.e., a total of 10 D3Ds) with kernel size of 3 and channel length of 64. When we perform 4x SR on Vid4 dataset (average resolution of HR is 700*500, batch size=1, input frames=7), the GPU memory cost is 2.5G. Then we replace D3Ds in D3Dnet by C3Ds, the GPU memory cost is 1.2G. It is worth noting that deformable operation actually increases the GPU memory cost for about 2.08 times.

YDDDDG commented 3 years ago

Thanks for your detailed reply!