Clarification on model parameters and input data handling for pansharpening task

[zhaomengjiao123]

Thank you for sharing this impressive model. After reviewing the code, I noticed that the model has a large number of parameters, and I would appreciate some clarification to ensure correct usage.

I intend to use this model for a pansharpening task. In my dataset, the LRMS images have a size of 32x32, while the PAN images are 128x128. Could you kindly advise whether I should perform a 4x upsampling of the LRMS images before feeding them into the model?

Additionally, I am unsure about how to configure the pt_img_size parameter in the model’s initialization. Given the input image sizes I mentioned, what would be the appropriate value for this setting?

Thank you very much for your time and guidance!

[294coder]

Hi, Zhao!

1) Param pt_img_size only take effect on RoPE and absolute PE, if you do not use it, it can ignore (and indeed, we do not use it as shown in its config). You can set to 128 if you like.

2) For your dataset, LRMS should be upsampled first to fit the PAN image size and further be concatenated in the model.

Cheers.

[294coder]

I will close this issue if you do not have any question. You can reopen it if you have any other questions.

[zhaomengjiao123]

Hi, Zhao!

1. Param only take effect on RoPE and absolute PE, if you do not use it, it can ignore (and indeed, we do not use it as shown in its config). You can set to 128 if you like.pt_img_size
2. For your dataset, LRMS should be upsampled first to fit the PAN image size and further be concatenated in the model.

Cheers. Hi ! I am using the following input data: LRMS: shape (B, 4, 128, 128) PAN: shape (B, 1, 128, 128) Here is the relevant portion of my Net class initialization:

def __init__(
self,
cfg, logger,
img_channel=4,
condition_channel=1,
out_channel=4,
width=32,
ssm_enc_blk_nums=[2,1,1],
middle_blk_nums=1,
ssm_dec_blk_nums=[2,1,1],
ssm_enc_convs= [[7, 11], [7, 11], [None, 11]],
ssm_dec_convs= [[None, 11], [7, 11], [7, 11]],
ssm_ratios=[2,2,1],
ssm_chan_upscale=[1,1,1],
ssm_enc_d_states= [[16, 32], [16, 32], [None, 32]],
ssm_dec_d_states= [[None, 32], [16, 32], [16, 32]],
window_sizes=[8,8, None],
upscale=4,
if_abs_pos=False,
if_rope=False,
pt_img_size=128,
drop_path_rate=0.1,
patch_merge=True,
):

Could you please confirm if these parameters are correctly set for my input data? Thank you for your assistance, and I appreciate your work on this project!

[294coder]

Hi, Zhao!

1. Param only take effect on RoPE and absolute PE, if you do not use it, it can ignore (and indeed, we do not use it as shown in its config). You can set to 128 if you like.pt_img_size

2. For your dataset, LRMS should be upsampled first to fit the PAN image size and further be concatenated in the model.

Cheers.

Hi !

I am using the following input data:

LRMS: shape (B, 4, 128, 128)

PAN: shape (B, 1, 128, 128)

Here is the relevant portion of my Net class initialization:

def __init__(

        self,

        cfg, logger,

        img_channel=4,

        condition_channel=1,

        out_channel=4,

        width=32,

        ssm_enc_blk_nums=[2,1,1],

        middle_blk_nums=1,

        ssm_dec_blk_nums=[2,1,1],

        ssm_enc_convs= [[7, 11], [7, 11], [None, 11]],

        ssm_dec_convs= [[None, 11], [7, 11], [7, 11]],

        ssm_ratios=[2,2,1],

        ssm_chan_upscale=[1,1,1],

        ssm_enc_d_states= [[16, 32], [16, 32], [None, 32]],

        ssm_dec_d_states= [[None, 32], [16, 32], [16, 32]],

        window_sizes=[8,8, None],

        upscale=4,

        if_abs_pos=False,

        if_rope=False,

        pt_img_size=128,

        drop_path_rate=0.1,

        patch_merge=True,

    ):

Could you please confirm if these parameters are correctly set for my input data?

Thank you for your assistance, and I appreciate your work on this project!

Hi,

Your configuration is correct. Try it to train your Pansharpening model.

Cheers!

[zhaomengjiao123]

Thank you for your prompt and helpful response regarding my previous inquiry. I truly appreciate your work on the LEMamba model; it is impressive and has been a great resource for my research. Wishing you all the best and continued success in your work!