search

ViTAE-Transformer / MTP

The official repo for [JSTARS'24] "MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining"
MIT License
168 stars 10 forks source link

Multichannel images #9

Open sebbelese opened 6 months ago

sebbelese commented 6 months ago

Hello,

It seems that the examples focus on 3 channel images.

Is semantic segmentation expected to work on multichannel images, and is there an example on that?

I know that mmsegmentation has LoadSingleRSImageFromFile, but I wonder if there would be any specific limitation associated with MTP? How to handle pre-trained weights?

Thanks!

DotWang commented 6 months ago

@sebbelese Theoretically, when in finetuning, the MTP weights can be used to understand any RS images, including multichannel images, even if MTP is not related to the spectral attribute.

To perform this with MMseg, you need to manually define all related operations in the pipeline, rather than only changing LoadSingleRSImageFromFile.

We have not presented related examples since they beyond the scope of this project.

sebbelese commented 6 months ago

Thanks for your reply.

By updating the dataloader and some parameters in the config file for model/preprocessing, I can run rvsa-l on on a semantic segmentation task with 13-channel input images. Initial results seem OK; I will wait for a longer training and see how it compare with other methods.

chartgod commented 5 months ago

Is it possible to perform the Semantic change detection function? If possible, can you tell me what to do?