测试结果图是灰色而不是彩色怎么解决？

[zhang-pan]

您好，非常感谢您的工作。我们通过运行test_IVF.py和test_MIF.py代码时，得到的结果图像都是类似灰度图的图像，也尝试修改代码中的data_IR=image_read_cv2(os.path.join(test_folder,"ir",img_name),mode='GRAY')[np.newaxis,np.newaxis, ...]/255.0灰度参数为RGB或者其他，但是都会报错。所以，请问这个应该怎么修改代码才能出现论文中的那种彩色图？感谢大佬指点

[yaoliu0803]

同问。请问这个问题解决了吗

[matrixgame2018]

以下是这个问题的解决思路

def bgr_to_ycrcb(path):
    one = cv2.imread(path,1)
    one = one.astype('float32')
    (B, G, R) = cv2.split(one)

    Y = 0.299 * R + 0.587 * G + 0.114 * B
    Cr = (R - Y) * 0.713 + 0.5
    Cb = (B - Y) * 0.564 + 0.5

    return Y, cv2.merge([Cr,Cb])

通过上述代码我们可以把YCBCR的Y代替Gray作为模型的input

I_VIS, CBCR = bgr_to_ycrcb(VIS_files[i])
I_VIS = np.expand_dims(I_VIS,axis=0)/255

在inference阶段 我们将得到的fi与上面的cv2.merge([Cr,Cb])重新组合得到新的YCBCR后，将新的YCBCR转化为RGB

def ycrcb_to_bgr(one):
    one = one.astype('float32')
    Y, Cr, Cb = cv2.split(one)
    B = (Cb - 0.5) * 1. / 0.564 + Y
    R = (Cr - 0.5) * 1. / 0.713 + Y
    G = 1. / 0.587 * (Y - 0.299 * R - 0.114 * B)
    return cv2.merge([B, G, R])

color = ycrcb_to_bgr(cv2.merge([fi,CBCR]))
img_save(color, img_name.split(sep='.')[0], test_out_folder)

最后附上效果展示的demo

[Zhaozixiang1228]

@matrixgame2018 Thank you very much for your answer. Your understanding is correct. @zhang-pan @yaoliu0803 The pipeline for this task is a single channel fusion. Subsequently, if it is a fusion of RGB with IV, according to the convention of the previous article, the RGB channel would be used to turn into a YCbCr space and then the Y channel would be used for fusion with IV. Then the RGB map is recovered by the new Y channel in conjunction with the original CbCr channel. However, direct end-to-end output of the three channels is fine, it just requires retraining the three-channel output network after modifying the output channels in CDDFuse.

[yuu-Wang]

以下是这个问题的解决思路

def bgr_to_ycrcb(path):
    one = cv2.imread(path,1)
    one = one.astype('float32')
    (B, G, R) = cv2.split(one)

    Y = 0.299 * R + 0.587 * G + 0.114 * B
    Cr = (R - Y) * 0.713 + 0.5
    Cb = (B - Y) * 0.564 + 0.5

    return Y, cv2.merge([Cr,Cb])

通过上述代码我们可以把YCBCR的Y代替Gray作为模型的input

I_VIS, CBCR = bgr_to_ycrcb(VIS_files[i])
I_VIS = np.expand_dims(I_VIS,axis=0)/255

在inference阶段 我们将得到的fi与上面的cv2.merge([Cr,Cb])重新组合得到新的YCBCR后，将新的YCBCR转化为RGB

def ycrcb_to_bgr(one):
    one = one.astype('float32')
    Y, Cr, Cb = cv2.split(one)
    B = (Cb - 0.5) * 1. / 0.564 + Y
    R = (Cr - 0.5) * 1. / 0.713 + Y
    G = 1. / 0.587 * (Y - 0.299 * R - 0.114 * B)
    return cv2.merge([B, G, R])

color = ycrcb_to_bgr(cv2.merge([fi,CBCR]))
img_save(color, img_name.split(sep='.')[0], test_out_folder)

最后附上效果展示的demo

抱歉，打扰了，我按照您的方法修改代码还是得不出彩色融合结果， def forward(self, inp_img): print(inp_img.shape) inp_enc_level1 = self.patch_embed(inp_img) out_enc_level1 = self.encoder_level1(inp_enc_level1) base_feature = self.baseFeature(out_enc_level1) detail_feature = self.detailFeature(out_enc_level1) return base_feature, detail_feature, out_enc_level1 Restormer_Encoder的forword打印显示输入inp_img是一个5维张量[1, 1, 256, 256, 3]，而Restormer_Encoder类的patch_embed方法期望输入是一个4维张量。不知有什么解决思路，谢谢再次分享！ 以下是运行MR-PET和MR-SPECT的错误代码： Traceback (most recent call last): File "/home/root3203/wanniny/code/compare exp/MMIF-CDDFuse-main/test_MIF.py", line 57, in feature_V_B, feature_V_D, feature_V = Encoder(data_VIS) File "/home/root3203/anaconda3/envs/python3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, kwargs) File "/home/root3203/anaconda3/envs/python3.9/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py", line 169, in forward return self.module(*inputs[0], *kwargs[0]) File "/home/root3203/anaconda3/envs/python3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "/home/root3203/wanniny/code/compare exp/MMIF-CDDFuse-main/net.py", line 359, in forward inp_enc_level1 = self.patch_embed(inp_img) File "/home/root3203/anaconda3/envs/python3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/home/root3203/wanniny/code/compare exp/MMIF-CDDFuse-main/net.py", line 332, in forward x = self.proj(x) File "/home/root3203/anaconda3/envs/python3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, **kwargs) File "/home/root3203/anaconda3/envs/python3.9/lib/python3.9/site-packages/torch/nn/modules/conv.py", line 463, in forward return self._conv_forward(input, self.weight, self.bias) File "/home/root3203/anaconda3/envs/python3.9/lib/python3.9/site-packages/torch/nn/modules/conv.py", line 459, in _conv_forward return F.conv2d(input, weight, bias, self.stride, RuntimeError: Expected 3D (unbatched) or 4D (batched) input to conv2d, but got input of size: [1, 1, 256, 256, 3]

[linzy88]

请问一下生成彩色图像结果是这样如何解决

[xiaboAAA]

请问一下生成彩色图像结果是这样如何解决

请问一下生成彩色图像结果是这样如何解决

你好 彩色图像是怎么生成的呢，是修改test还是在net里就要修改通道呢

[Tianfeng-Pink]

请问一下生成彩色图像结果是这样如何解决

您好，请问你这个问题解决了吗？我也碰到了类似的问题

[linzy88]

请问一下生成彩色图像结果是这样如何解决

请问一下生成彩色图像结果是这样如何解决

你好 彩色图像是怎么生成的呢，是修改test还是在net里就要修改通道呢

在test中修改，将Y代替Gray作为输入

[linzy88]

请问一下生成彩色图像结果是这样如何解决

您好，请问你这个问题解决了吗？我也碰到了类似的问题

我将最后保存图片的img_save()函数中“image = image.astype(np.uint8)”这行代码注释掉，就解决了。

[Tianfeng-Pink]

请问一下生成彩色图像结果是这样如何解决

您好，请问你这个问题解决了吗？我也碰到了类似的问题

我将最后保存图片的img_save()函数中“image = image.astype(np.uint8)”这行代码注释掉，就解决了。

请问能贴一下原代码或者说一下在哪个文件内吗？我找了 utils/img_read_save.py 下面的 img_save() 函数 ，好像本来就没有 “image = image.astype(np.uint8)” 这一行代码

[hell712]

同问。请问这个问题解决了 你的问题解决了吗？我按照这个思路加进去还是出不来结果

[Hydrageno]

同问。请问这个问题解决了 你的问题解决了吗？我按照这个思路加进去还是出不来结果

我用下面的代码好像解决了问题： 在test_IVF.py的for循环中：

           .....
            data_IR=image_read_cv2(os.path.join(test_folder,"ir",img_name),mode='GRAY')[np.newaxis,np.newaxis, ...]/255.0
            data_VIS = cv2.split(image_read_cv2(os.path.join(test_folder,"vi",img_name), mode='YCrCb'))[0][np.newaxis,np.newaxis, ...]/255.0

           # ycrcb, uint8
            data_VIS_BGR = cv2.imread(os.path.join(test_folder,"vi",img_name))
            _, data_VIS_Cr, data_VIS_Cb = cv2.split(cv2.cvtColor(data_VIS_BGR, cv2.COLOR_BGR2YCrCb))

            data_IR,data_VIS = torch.FloatTensor(data_IR),torch.FloatTensor(data_VIS)
            ......
            fi = np.squeeze((data_Fuse * 255.0).cpu().numpy())

           # float32 to uint8
            fi = fi.astype(np.uint8)
           # concatnate
            ycrcb_fi = np.dstack((fi, data_VIS_Cr, data_VIS_Cb))
            rgb_fi = cv2.cvtColor(ycrcb_fi, cv2.COLOR_YCrCb2RGB)
            img_save(rgb_fi, img_name.split(sep='.')[0], test_out_folder)

[NUTSHELL-X]

同问。请问这个问题解决了 你的问题解决了吗？我按照这个思路加进去还是出不来结果

感谢 Hydrageno ，在他的基础上，我目前尝试得到的方法：要想得到彩色的医学图像，在test_MIF.py中，将with torch.no_grad():部分替换为以下内容

with torch.no_grad():
            for img_name in os.listdir(os.path.join(test_folder,dataset_name.split('_')[0])):
                data_IR=image_read_cv2(os.path.join(test_folder,dataset_name.split('_')[0],img_name),mode='GRAY')[np.newaxis,np.newaxis, ...]/255.0
                data_VIS = cv2.split(image_read_cv2(os.path.join(test_folder,dataset_name.split('_')[1],img_name), mode='YCrCb'))[0][np.newaxis,np.newaxis, ...]/255.0

                data_VIS_BGR = cv2.imread(os.path.join(test_folder,dataset_name.split('_')[1],img_name))
                _, data_VIS_Cr, data_VIS_Cb = cv2.split(cv2.cvtColor(data_VIS_BGR, cv2.COLOR_BGR2YCrCb))
                data_IR,data_VIS = torch.FloatTensor(data_IR),torch.FloatTensor(data_VIS)
                data_VIS, data_IR = data_VIS.cuda(), data_IR.cuda()

                feature_V_B, feature_V_D, feature_V = Encoder(data_VIS)
                feature_I_B, feature_I_D, feature_I = Encoder(data_IR)
                feature_F_B = BaseFuseLayer(feature_V_B + feature_I_B)
                feature_F_D = DetailFuseLayer(feature_V_D + feature_I_D)
                if ckpt_path==CDDFuse_path:
                    data_Fuse, _ = Decoder(data_IR+data_VIS, feature_F_B, feature_F_D)
                else:
                    data_Fuse, _ = Decoder(None, feature_F_B, feature_F_D)
                data_Fuse=(data_Fuse-torch.min(data_Fuse))/(torch.max(data_Fuse)-torch.min(data_Fuse))
                fi = np.squeeze((data_Fuse * 255).cpu().numpy())
                fi=fi.astype(np.uint8)
                ycrcb_fi = np.dstack((fi, data_VIS_Cr, data_VIS_Cb))
                rgb_fi = cv2.cvtColor(ycrcb_fi, cv2.COLOR_YCrCb2RGB)
                img_save(rgb_fi, img_name.split(sep='.')[0], test_out_folder)

[qqbangbangbang]

@zhang-pan @Tianfeng-Pink @Zhaozixiang1228 @matrixgame2018 @Hydrageno 前辈们您们好，我在测试时候显示OSError: cannot write mode F as PNG ，您有遇到过吗，麻烦您可以给点意见 File "test_IVF.py", line 53, in img_save(fi, img_name.split(sep='.')[0], test_out_folder) File "/root/autodl-tmp/MMIF-CDDFuse-main/utils/img_read_save.py", line 21, in img_save imsave(os.path.join(savepath, "{}.png".format(imagename)),image) File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/skimage/io/_io.py", line 143, in imsave return call_plugin('imsave', fname, arr, plugin=plugin, *plugin_args) File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/skimage/io/manage_plugins.py", line 207, in call_plugin return func(args, kwargs) File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/imageio/v2.py", line 397, in imwrite return file.write(im, kwargs) File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/imageio/core/v3_plugin_api.py", line 367, in exit self.close() File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/imageio/plugins/pillow.py", line 144, in close self._flush_writer() File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/imageio/plugins/pillow.py", line 485, in _flush_writer primary_image.save(self._request.get_file(), **self.save_args) File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/PIL/Image.py", line 2439, in save save_handler(self, fp, filename) File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/PIL/PngImagePlugin.py", line 1282, in _save raise OSError(msg) from e OSError: cannot write mode F as PNG

[LiAnwen99]

@NUTSHELL-X Hello, but the image generated by your code looks like this. Did you do any post-processing?

[NUTSHELL-X]

@NUTSHELL-X Hello, but the image generated by your code looks like this. Did you do any post-processing?

I didn't do any post-processing, if your generated image is weird, it is recommanded to see how your fused gray image looks like(fi in the code).Maybe something is wrong before colorized.

[FMVPJet]

同问。请问这个问题解决了 你的问题解决了吗？我按照这个思路加进去还是出不来结果

感谢 Hydrageno ，在他的基础上，我目前尝试得到的方法：要想得到彩色的医学图像，在test_MIF.py中，将with torch.no_grad():部分替换为以下内容

with torch.no_grad():
            for img_name in os.listdir(os.path.join(test_folder,dataset_name.split('_')[0])):
                data_IR=image_read_cv2(os.path.join(test_folder,dataset_name.split('_')[0],img_name),mode='GRAY')[np.newaxis,np.newaxis, ...]/255.0
                data_VIS = cv2.split(image_read_cv2(os.path.join(test_folder,dataset_name.split('_')[1],img_name), mode='YCrCb'))[0][np.newaxis,np.newaxis, ...]/255.0

                data_VIS_BGR = cv2.imread(os.path.join(test_folder,dataset_name.split('_')[1],img_name))
                _, data_VIS_Cr, data_VIS_Cb = cv2.split(cv2.cvtColor(data_VIS_BGR, cv2.COLOR_BGR2YCrCb))
                data_IR,data_VIS = torch.FloatTensor(data_IR),torch.FloatTensor(data_VIS)
                data_VIS, data_IR = data_VIS.cuda(), data_IR.cuda()

                feature_V_B, feature_V_D, feature_V = Encoder(data_VIS)
                feature_I_B, feature_I_D, feature_I = Encoder(data_IR)
                feature_F_B = BaseFuseLayer(feature_V_B + feature_I_B)
                feature_F_D = DetailFuseLayer(feature_V_D + feature_I_D)
                if ckpt_path==CDDFuse_path:
                    data_Fuse, _ = Decoder(data_IR+data_VIS, feature_F_B, feature_F_D)
                else:
                    data_Fuse, _ = Decoder(None, feature_F_B, feature_F_D)
                data_Fuse=(data_Fuse-torch.min(data_Fuse))/(torch.max(data_Fuse)-torch.min(data_Fuse))
                fi = np.squeeze((data_Fuse * 255).cpu().numpy())
                fi=fi.astype(np.uint8)
                ycrcb_fi = np.dstack((fi, data_VIS_Cr, data_VIS_Cb))
                rgb_fi = cv2.cvtColor(ycrcb_fi, cv2.COLOR_YCrCb2RGB)
                img_save(rgb_fi, img_name.split(sep='.')[0], test_out_folder)

倒数第二步，转为BGR颜色才正确 rgb_fi = cv2.cvtColor(ycrcb_fi, cv2.COLOR_YCrCb2BGR)

[FMVPJet]

@zhang-pan @Tianfeng-Pink @Zhaozixiang1228 @matrixgame2018 @Hydrageno 前辈们您们好，我在测试时候显示OSError: cannot write mode F as PNG ，您有遇到过吗，麻烦您可以给点意见 File "test_IVF.py", line 53, in img_save(fi, img_name.split(sep='.')[0], test_out_folder) File "/root/autodl-tmp/MMIF-CDDFuse-main/utils/img_read_save.py", line 21, in img_save imsave(os.path.join(savepath, "{}.png".format(imagename)),image) File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/skimage/io/_io.py", line 143, in imsave return call_plugin('imsave', fname, arr, plugin=plugin, *plugin_args) File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/skimage/io/manage_plugins.py", line 207, in call_plugin return func(args, kwargs) File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/imageio/v2.py", line 397, in imwrite return file.write(im, kwargs) File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/imageio/core/v3_plugin_api.py", line 367, in exit self.close() File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/imageio/plugins/pillow.py", line 144, in close self._flush_writer() File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/imageio/plugins/pillow.py", line 485, in _flush_writer primary_image.save(self._request.get_file(), **self.save_args) File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/PIL/Image.py", line 2439, in save save_handler(self, fp, filename) File "/root/miniconda3/envs/cddfuse/lib/python3.8/site-packages/PIL/PngImagePlugin.py", line 1282, in _save raise OSError(msg) from e OSError: cannot write mode F as PNG

img_read_save.py 里的img_save 改成cv保存 ` def img_save(image,imagename,savepath): if not os.path.exists(savepath): os.makedirs(savepath) cv2.imwrite(os.path.join(savepath, "{}.png".format(imagename)),image) '

[GShef]

Hello, I used the code above to retain color information in the fused image. My resulting fused images look like this:

I am using a different dataset with Infrared and Visible image pairs, but applying the CDDFuse model with the exact same modified version of test_IVF.py posted on this thread.