icon-lab / ResViT

Official Implementation of ResViT: Residual Vision Transformers for Multi-modal Medical Image Synthesis
Other
133 stars 26 forks source link

Encountering Runtime Errors with Color Image-to-Image Translation with Custom Channel Configuration #21

Open NuclearKAI opened 8 months ago

NuclearKAI commented 8 months ago

Hello,

I am working on a project using your resvit model, and I have encountered a series of runtime errors related to the configuration of input and output channels. My goal is to train a model with custom input (--input_nc) and output (--output_nc) channel configurations, but I am running into issues. and i want to train image-to-image translation project using resvit, aiming to train a model for color paired image translation from domain A to domain B (both A and B are sets of color images). I intend to use --input_nc 3 and --output_nc 3 to handle RGB images for both input and output. However, I've encountered runtime errors related to channel configuration that I cannot seem to resolve.

Environment:

Issue Description: I can successfully train my model when both --input_nc and --output_nc are set to 1, which results in processing grayscale images. However, I need to train my model with color images (3 channels) and potentially different numbers of output channels. Whenever I try to adjust the --input_nc and --output_nc values away from 1, I encounter runtime errors indicating a mismatch between the expected and actual number of input channels to certain convolutional layers.

For example, when attempting to train with color images as input, I adjusted the parameters to --input_nc 3 and --output_nc 1, and received the following error:

RuntimeError: Given groups=1, weight of size [64, 4, 7, 7], expected input[1, 3, 262, 262] to have 4 channels, but got 3 channels instead

Similarly, adjusting the parameters to other values results in errors like:

RuntimeError: Given groups=1, weight of size [64, 6, 4, 4], expected input[1, 4, 256, 256] to have 6 channels, but got 4 channels instead

Any advice or guidance on resolving this channel configuration issue would be greatly appreciated. Thank you very much for your time and assistance!

Best regards