microsoft / satclip

PyTorch implementation of SatCLIP
MIT License
226 stars 22 forks source link

Question about 13th band #16

Closed PlekhanovaElena closed 4 months ago

PlekhanovaElena commented 4 months ago

Hi there,

I've noticed that there is in the code of transforms.py in the get_pretrained_s2_train_transform function there is an imput of 0s-filled B10 band:

B10 = np.zeros((1, *image.shape[1:]), dtype=image.dtype)
image = np.concatenate([image[:10], B10, image[10:]], axis=0)

I'm just curious - why do you do this?

Kind regards, Elena

konstantinklemmer commented 4 months ago

The SSL4EO vision encoders we use are pretrained on 13 channels (https://torchgeo.readthedocs.io/en/stable/api/models.html#sentinel-2), but our S2-100K inputs are just 12 channels so we zero-pad one channel.

PlekhanovaElena commented 4 months ago

Aha, got it, thank you for the explanation!