How to convert the prediction_R to a RGB image?

[linlanye]

I want to show the reflectance image correctlly. How to proccess the prediction_R got by prediction_S, prediction_R, rgb_s = self.netG.forward(input_images)?

Thanks for your help!

[zhangmohan]

I have the same question, have you solved it?

[linlanye]

no

[sabtorres]

We're trying to use this code here and ran into the same issue, does anyone have an answer yet?

I know that prediction_S and prediction_R are two four dimensional tensors and each dimension probably means each one of the 16 images in a batch, the width, the height and each one of the 3 color channels, but they seem to be transformed in some way I can't quite decipher yet . Apparently some of those transformations are undone in the JointLoss class (in models.networks.py)

` def call(self, input_images, prediction_S, prediction_R, _rgb_s, targets, data_set_name, epoch): num_images = prediction_S.size(0) # must be even number mid_point = num_images/2

    # undo gamma correction
    prediction_S = prediction_S / 0.4545
    prediction_R = prediction_R / 0.4545
    _rgb_s = _rgb_s / 0.4545

    rgb_s = _rgb_s.unsqueeze(2)
    rgb_s = rgb_s.unsqueeze(3)
    rgb_s = rgb_s.repeat(1,1, prediction_S.size(2), prediction_S.size(3))

    prediction_Sr = prediction_S.repeat(1,3,1,1)
    prediction_Sr = prediction_Sr + rgb_s

    # downsample using bilinear inteporlation
    # down_sample = nn.AvgPool2d(2, stride=2)
    # prediction_S_1 = down_sample(prediction_S)
    # prediction_S_2 = down_sample(prediction_S_1)
    # prediction_S_3 = down_sample(prediction_S_2)

    # downsample using NN
    prediction_S_1 = prediction_S[:,:,::2,::2]
    prediction_S_2 = prediction_S_1[:,:,::2,::2]
    prediction_S_3 = prediction_S_2[:,:,::2,::2]

    rc_loss =  self.w_rc * self.MultiViewRefConsistencyLoss(prediction_R,targets)
    rl_loss =  self.w_cy * self.MultiViewRelightingLoss(input_images, prediction_R, prediction_Sr, targets)

    # multi-scale shading smoothness term
    ss_loss =  self.w_ss  * self.LocalShadSmoothenessLoss(prediction_S, targets,0)
    ss_loss = ss_loss + 0.5 * self.w_ss * self.LocalShadSmoothenessLoss(prediction_S_1, targets,1)
    ss_loss = ss_loss + 0.3333 * self.w_ss  * self.LocalShadSmoothenessLoss(prediction_S_2, targets,2)
    ss_loss = ss_loss + 0.25 * self.w_ss   * self.LocalShadSmoothenessLoss(prediction_S_3, targets,3)

    # spatial-temporal densely connected smoothness term
    rs_loss =  self.w_rs_dense * self.SpatialTemporalBilateralRefSmoothnessLoss(prediction_R, targets, 'R' ,5)
    shading_color_loss = self.w_regularization * self.ShadingPenaltyLoss(prediction_S)

    print("ss loss", ss_loss.data[0])
    print("rs_loss", rs_loss.data[0])
    print("rl_loss loss", rl_loss.data[0])
    print("rc_loss loss", rc_loss.data[0])
    print("regularization loss ", shading_color_loss.data[0])

    total_loss = ss_loss + rs_loss + rl_loss  + rc_loss + shading_color_loss

    self.total_loss = total_loss

    return total_loss.data[0]

`

We're currently studying this code and the related paper in my lab so I'll probably make some progress soon and will post it here for you guys if I do. My guess is it shouldn't be that complicated since they show those reflectance and shading images in the paper :)

[sabtorres]

Keep in mind also as they stated at the start of section 5 in the paper, they deal with the images in the logarithmic domain, so maybe we also need to undo this logarithm.

[chester256]

Did you solve this problem?