WIP implementation for patch level world cover embeddings

[lillythomas]

Relates to https://github.com/orgs/Clay-foundation/discussions/291

[lauracchen]

Hi @lillythomas ! Thanks so much for this. Would you also be able to update the worldcover script to include the option to generate embeddings with all bands included (not just partial inputs)? The app team is wondering if generating with only a couple bands is having an impact on performance.

cc @antymoro @lukaszknasiecki

[lillythomas]

Hi @lauracchen sure thing!

[lillythomas]

Actually @lauracchen it appears the Worldcover composite we are using is just RGB-Nir (https://esa-worldcover.org/en/data-access). There is a SWIR composite provided as well, which we can stack on the RGB-Nir, but this still misses out on 4/10 Sentinel-2 inputs. We can similarly stack the Sentinel-1 composite as well.

[lillythomas]

Patch-level sim search with the currently-in-place RGB-Nir worldcover model is working, albeit the results are still a bit random:

Example 1:

Example 2:

Working on the adjustments per https://github.com/Clay-foundation/model/pull/231#issuecomment-2079780410

[antymoro]

hi @lillythomas

if we stack these 4 files, I think we might cover for all sentinel-2 bounds... correct?

https://esa-worldcover-s2.s3.amazonaws.com/swir/2020/N55/ESA_WorldCover_10m_2020_v100_N55E014_SWIR.tif https://esa-worldcover-s2.s3.amazonaws.com/ndvi/2020/N55/ESA_WorldCover_10m_2020_v100_N55E014_NDVI.tif https://esa-worldcover-s1.s3.amazonaws.com/gamma0/2020/N55/ESA_WorldCover_10m_2020_v100_N55E014_GAMMA0.tif https://esa-worldcover-s1.s3.amazonaws.com/vvvhratio/2020/N55/ESA_WorldCover_10m_2020_v100_N55E014_S1VVVHratio.tif

[lillythomas]

@antymoro I've implemented the concatenation of the RGB-Nir composite with the SWIR and SAR composites (see the new script run_msi_sar.py). Feel free to try it on your end.