nasaharvest / presto

Lightweight, Pre-trained Transformers for Remote Sensing Timeseries
https://arxiv.org/abs/2304.14065
MIT License
151 stars 26 forks source link

Update EuroSat eval task to use splits from https://arxiv.org/pdf/1911.06721.pdf #8

Closed gabrieltseng closed 9 months ago

gabrieltseng commented 1 year ago

This PR updates the EuroSat splits so that they reflect the splits used by In-domain representation learning for remote sensing.

These are the splits also used by TorchGeo, allowing for easier comparison to TorchGeo models.

The updated accuracy results (Table 5 from the paper) are below. The table compares the original splits (what is currently in the paper) with the new splits (in this PR):

Model # input pixels k=5 k=20 k=100
ScaleMAE @ 16 256 0.729 0.727 0.695
SatMAE @ 16 256 0.723 0.721 0.676
Presto (RGB), original splits 9 0.824 0.804 0.752
Presto (MS), original splits 9 0.864 0.841 0.795
Presto (RGB), new splits 9 0.815 0.800 0.760
Presto (MS), new splits 9 0.860 0.840 0.794
gabrieltseng commented 9 months ago

Superseded by #14