Vandermode / ELD

Physics-based Noise Modeling for Extreme Low-light Photography (CVPR 2020 Oral & TPAMI 2021)
http://openaccess.thecvf.com/content_CVPR_2020/html/Wei_A_Physics-Based_Noise_Formation_Model_for_Extreme_Low-Light_Raw_Denoising_CVPR_2020_paper.html
MIT License
485 stars 66 forks source link

Questions about released camera_params #24

Open sky135410 opened 2 years ago

sky135410 commented 2 years ago

Hi I have some questions about released camera parameters For example: In NikonD850_params.npy, it has key name like Profile-1(G_scale, g_scale, R_scale), Profile-2, G_shape. I don't know they are corresponded to which parameter listed in paper table1 respectively.

Another question, are these noise parameters the same in R, G, B channel of raw? (like system gain K, tukey lambda shape and sigma, row noise sigma)

happycaoyue commented 2 years ago

G_scale and R_scale denote Scale sigma_TL and sigma_r in Table 1. g_scale denotes Gaussian noise.

The noise parameters in R, G, B channel of raw are the same in ELD noise model.

sky135410 commented 2 years ago

G_scale and R_scale denote Scale sigma_TL and sigma_r in Table 1. g_scale denotes Gaussian noise.

The noise parameters in R, G, B channel of raw are the same in ELD noise model.

Thanks for your reply. You say g_scale denotes Gaussian noise, but in table 1. doesn't have this part. (g_scale also has bias, sigma, slope in NikonD850_params.npy).

Other question is that what G_shape denotes in .npy ?

Besides, why in NikonD850_params.npy has Profile-1 and Profile-2?

Vandermode commented 2 years ago

Other question is that what G_shape denotes in .npy ?

A: The shape param (\lambda) in TL distribution.

why in NikonD850_params.npy has Profile-1 and Profile-2?

A: just a historical development issue (you can view it as two instantiations of parameter calibration). Please carefully read the paper before asking further questions, thx.

sky135410 commented 2 years ago

Because paper doesn't explain about the camera parameters in npy file and the parameter's names are different between npy file and the description in paper, so I have a lot of problems.

G_shape denotes The shape param (\lambda) in TL distribution that means in different ISO(or K) has different TL shape. Whether I understand right or not? If right, I am confused how to plot the relation between sigma_TL and K like Figure 9. in the paper, because different K has different TL shape and sigma_TL.

Vandermode commented 2 years ago

Because paper doesn't explain about the camera parameters in npy file and the parameter's names are different between npy file and the description in paper, so I have a lot of problems.

But it should be easy to be inferred yourself. Please note the implementation of calibration method is made confidential currently, so I would not reveal too much about it.

G_shape denotes The shape param (\lambda) in TL distribution that means in different ISO(or K) has different TL shape. Whether I understand right or not?

You may randomly sample G_shape, or just choose one from them. They are not correlated with K

sky135410 commented 2 years ago

But it should be easy to be inferred yourself. Please note the implementation of calibration method is made confidential currently, so I would not reveal too much about it.

Thanks for your detailed answers. I know there are confidential details, if can not reveal the method, just let me know 👍

You may randomly sample G_shape, or just choose one from them. They are not correlated with K

So If I just choose one G_shape, for example lambda=0.14 that approximating to N(0, 2.142), the sigma_TL is a constant value. Thus the joint distribution of (K, sigma_TL) in Fig 9 of paper will become a horizontal line. Is it correct?

Vandermode commented 2 years ago

So If I just choose one G_shape, for example lambda=0.14 that approximating to N(0, 2.142), the sigma_TL is a constant value. Thus the joint distribution of (K, sigma_TL) in Fig 9 of paper will become a horizontal line. Is it correct?

For lambda=0.14, TL(0.14, 0, \sigma) approximates N(0, 2.142\sigma). Note \sigma here is not standard deviation, instead it terms "scale param"

sky135410 commented 2 years ago

For lambda=0.14, TL(0.14, 0, \sigma) approximates N(0, 2.142\sigma). Note \sigma here is not standard deviation, instead it terms "scale param"

Thanks for your answer. I understand the sigma_TL now. I still have a question that is if different K has different lambda, is it reasonable to randomly choose a lambda shape and plot the joint distribution of (K, sigma_TL)? or should I how to plot the joint distribution of (K, sigma_TL) in Fig 9.

zhongyi-zhou commented 2 years ago

Hi all, Thanks for the informative discussions! I also want to generate another dataset based on the camera param available! (Thanks again to the authors who eventually decided to share it!)

Just want to confirm, here is how I implemented the "R" noise in Table 1:

The following codes are inserted here

        if 'r' in self.model:
            z = self.unpack_bayer(z)
            z = z + np.tile(np.random.randn(z.shape[0])[:,np.newaxis], (1, z.shape[1])).astype(np.float32) * np.maximum(sigma_r, 1e-10)
            z = self.pack_bayer(z)

These codes are inserted here

        log_r = np.random.standard_normal() * camera_params['R_scale']['sigma'] * 1 +\
             camera_params['R_scale']['slope'] * log_K + camera_params['R_scale']['bias']
        sigma_r = np.exp(log_r)

and pack_bayer and unpack_bayer are defined as:


    def unpack_bayer(self, x):
        _, h, w = x.shape
        H = h*2
        W = w*2
        out = np.zeros((H, W))

        out[0:H:2, 0:W:2] = x[0]
        out[0:H:2, 1:W:2] = x[1]
        out[1:H:2, 1:W:2] = x[2]
        out[1:H:2, 0:W:2] = x[3]
        return out

    def pack_bayer(self, raw):
        h, w = raw.shape
        out = np.concatenate((raw[np.newaxis, 0:h:2, 0:w:2],
                              raw[np.newaxis, 0:h:2, 1:w:2],
                              raw[np.newaxis, 1:h:2, 1:w:2],
                              raw[np.newaxis, 1:h:2, 0:w:2]), axis=0)

Based on these codes I've generated a figure similar to Figure 5 which seems to have such row-by-row noise features.

Could you kindly check whether the codes are correct?

sky135410 commented 2 years ago

Based on these codes I've generated a figure similar to Figure 5 which seems to have such row-by-row noise features.

Could you kindly check whether the codes are correct?

I think your row noise gereration codes are correct.