Algolzw / BSRT

Pytorch code for "BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment", CVPRW, 1st place in NTIRE 2022 BurstSR Challenge (real-world track).
MIT License
179 stars 13 forks source link

Get flow between frames ref and other #10

Open qtct opened 1 year ago

qtct commented 1 year ago

flows_list = [flow.view(b, n, 2, h // (2 (i)), w // (2 (i))) for flow, i in zip(flows, range(3))] RuntimeError: shape '[4, 7, 2, 64, 64]' is invalid for input of size 8192

In the get_ref_flows function of bsrt.py,I had such a problem without any modification,what's wrong,thanks for your response.

Algolzw commented 1 year ago

Hi, Does this problem happen during training or testing? Can also you print the flow sizes and x_ref's size?

qtct commented 1 year ago

Thanks for your reply, The flow size is [28,2,64,64] and the x_ref size is [28,3,64,64]. One more thing,I changed the way the optical downsampled like above two pictures,does it also right? And I wonder the x_ref should be x_ref = x[:, self.center_frame_idx, :, :, :].repeat(1, n, 1, 1, 1).reshape(-1,c,h,w) rather than x_ref = x[:, self.center_frame_idx:self.center_frame_idx +1, :, :,:].repeat(1, n, 1, 1, 1).reshape(-1,c,h,w). That is my humble opinion.

------------------ 原始邮件 ------------------ 发件人: @.>; 发送时间: 2023年3月5日(星期天) 晚上8:37 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [Algolzw/BSRT] Get flow between frames ref and other (Issue #10)

Hi, Can also you print the flow sizes and x_ref's size?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

Algolzw commented 1 year ago

Aha. If your first flow size is [28, 2, 64, 64], it should be well reshaped to [4, 7, 2, 64, 64]. But where do you get the input size of 8192?

Here I use self.center_frame_idx:self.center_frame_idx +1 to keep the dim to perform the following repeat operation.

qtct commented 1 year ago

Thanks for your reply. I found the reason is that my spynet had not been applied correctly. Does your DCNv2 code support Cuda 11?

------------------ 原始邮件 ------------------ 发件人: "Algolzw/BSRT" @.>; 发送时间: 2023年3月5日(星期天) 晚上10:50 @.>; @.**@.>; 主题: Re: [Algolzw/BSRT] Get flow between frames ref and other (Issue #10)

Aha. If your first flow size is [28, 2, 64, 64], it should be reshaped well to [4, 7, 2, 64, 64]. But where do you get the input size of 8192?

Here I use self.center_frame_idx:self.center_frame_idx +1 to keep the dim to perform the following repeat operation.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

Algolzw commented 1 year ago

Sure, you can refer to this repo to install DCNv2 for CUDA 11: https://github.com/Algolzw/BSRT/issues/9#issuecomment-1445994347