ddlee-cn / MuLUT

[ECCV 2022 & T-PAMI 2024] Multiple Look-Up Tables for Efficient Image Restoration
https://mulut.pages.dev
MIT License
90 stars 9 forks source link

A little question about training #14

Closed Qinluyi closed 1 month ago

Qinluyi commented 1 month ago

if s + 1 == stages: avg_factor, bias, norm = len(modes), 0, 1 x = round_func((pred / avg_factor) + bias) if phase == "train": x = x / 255.0

In the training stage,when it comes to the last stage, why the bias and norm are 0 and 1 respectively,instead of 127 and 255 in previous stages? Besides, why the avg_factor isn't len(modes)4 just like it is in previous stage? It rotate 4 times in every mode. I think len(modes)4 make sense. Could you please explain these for me?

ddlee-cn commented 1 month ago

if s + 1 == stages: avg_factor, bias, norm = len(modes), 0, 1 x = round_func((pred / avg_factor) + bias) if phase == "train": x = x / 255.0

In the training stage,when it comes to the last stage, why the bias and norm are 0 and 1 respectively,instead of 127 and 255 in previous stages? Besides, why the avg_factor isn't len(modes)4 just like it is in previous stage? It rotate 4 times in every mode. I think len(modes)4 make sense. Could you please explain these for me?

Thanks for your interest in our work.

We followed SR-LUT in this one. Empirically, it is indeed better than the variant with avg_factor. From my point of view, since there is a direct supervision from the ground truth pixel values (0-1) for the second stage, omitting averaging helps the variety of predictions across different rotations and modes, without worrying about training stability like in the intermediate stages. Hope it helps.

Qinluyi commented 1 month ago

Thank you for your answer, it helps a lot.

Qinluyi commented 1 month ago

Besides,I wonder if there is a little problem that the "self.conv6 = Conv(nf 5, 1 upscale upscale, 1) " should be "self.conv6 = Conv(nf 5, out_c upscale upscale, 1)" in the MuLUTUnit module in network.py , so that the input parameter out_c can be used. In the DMNet module, you use "self.model = MuLUTUnit('2x2', nf, upscale=2, out_c=3, dense=dense)",the out_c is 3,so in the MuLUTUnit module, the self.conv6 should produce output with out_c upscale upscale channels, and in the next step you use pixelshuffle(upscale) and the output will have dimension: batchsize,out_c,Hupscale,Wupscale. Am I right? looking forward to your answer.

ddlee-cn commented 1 month ago

Besides,I wonder if there is a little problem that the "self.conv6 = Conv(nf 5, 1 upscale upscale, 1) " should be "self.conv6 = Conv(nf 5, out_c upscale upscale, 1)" in the MuLUTUnit module in network.py , so that the input parameter out_c can be used. In the DMNet module, you use "self.model = MuLUTUnit('2x2', nf, upscale=2, out_c=3, dense=dense)",the out_c is 3,so in the MuLUTUnit module, the self.conv6 should produce output with out_c upscale upscale channels, and in the next step you use pixelshuffle(upscale) and the output will have dimension: batchsize,out_c,H_upscale,W_upscale. Am I right? looking forward to your answer.

Yeah, you can change it to support multi-channel output.

Qinluyi commented 1 month ago

Thank you for your reply.