search

zhengchen1999 / DAT

PyTorch code for our ICCV 2023 paper "Dual Aggregation Transformer for Image Super-Resolution"
Apache License 2.0
391 stars 37 forks source link

Is it possible to feed two concatenated input images to this model #61

Open cs-7700 opened 3 weeks ago

cs-7700 commented 3 weeks ago

Hello

I am wondering if i can feed two concatenated images instead of one into your model, with size of 256*1024. is it possible to changes the input channel from 3 to 6?

waiting for your response please.

@zhengchen1999 @yulunzhang

zhengchen1999 commented 3 weeks ago

This requires modifying the model implementation and retraining the model.

cs-7700 commented 2 weeks ago

This requires modifying the model implementation and retraining the model.

even if i did horizontal concatenation? and keep the in_channels=3? or it will be possible with these setting? @zhengchen1999

zhengchen1999 commented 2 weeks ago

Horizontal connection can be input without retraining. However, the results cannot be guaranteed. In addition, this is equivalent to upsacle two images at the same time, which is not much different from upsacling them one by one.

cs-7700 commented 2 weeks ago

Horizontal connection can be input without retraining. However, the results cannot be guaranteed. In addition, this is equivalent to upsacle two images at the same time, which is not much different from upsacling them one by one.

So, what is your suggestions for making the model accept 2 images instead of single? @zhengchen1999

zhengchen1999 commented 2 weeks ago

I don't know your purpose. If it's for upscaling two images, then input them separately. If it's for image fusion, change in_channels to 6 and retrain the model (on your dataset)."

cs-7700 commented 2 weeks ago

I don't know your purpose. If it's for upscaling two images, then input them separately. If it's for image fusion, change in_channels to 6 and retrain the model (on your dataset)."

Yes i want it for image fusion... if the in_channels 6, the out_channels will be the same 6? becuase there no out_channel attribute!!!

because i change the in_channels to 6 but facing problem with the result image...

@zhengchen1999

zhengchen1999 commented 2 weeks ago

Modify here: https://github.com/zhengchen1999/DAT/blob/main/basicsr/archs/dat_arch.py#L748

cs-7700 commented 1 week ago

Modify here: https://github.com/zhengchen1999/DAT/blob/main/basicsr/archs/dat_arch.py#L748

My dataset is like following:

Input: two RGB images of size 256x1024 concatenated in channel dimension --> so 6 channels , 256x1024 width / height output image: RGB image of size 256x1024x3

Can I train the model with this image sizes or the image size needs to be square for training?

@zhengchen1999 @yulunzhang

zhengchen1999 commented 1 week ago

It can be used; square input is just a standard setting. Our method can accept inputs of any size.