search

NASA-IMPACT / hls-foundation-os

This repository contains examples of fine-tuning Harmonized Landsat and Sentinel-2 (HLS) Prithvi foundation model.
Apache License 2.0
319 stars 83 forks source link

Benchmarks #14

Closed Spiruel closed 1 year ago

Spiruel commented 1 year ago

Hi,

Are there any benchmark results for the finetuning efforts? I would like to know eg. accuracy for crop type classification on the HSL data.

Cheers

thesujitroy commented 1 year ago
There is none on the benchmark dataset. However, the model has been tested with Unet architecture for comparison. Below is the result: Class Accuracy IoU Precision Recall F1 Score
Natural Vegetation 0.63667133 0.457776506 0.619654332 0.63667133 0.628047583
Forest 0.717167742 0.477231243 0.587873474 0.717167742 0.646115827
Corn 0.633266833 0.522647144 0.749499561 0.633266833 0.686498045
Soybeans 0.66765162 0.51677767 0.695758167 0.66765162 0.681415187
Wetlands 0.603593075 0.4109703 0.562898078 0.603593075 0.58253572
Developed/Barren 0.602285469 0.463743295 0.668438842 0.602285469 0.633640197
Open Water 0.8775865 0.75960682 0.849630864 0.8775865 0.863382446
Winter Wheat 0.663901059 0.495049223 0.660609926 0.663901059 0.662251404
Alfalfa 0.590268065 0.384786488 0.525017524 0.590268065 0.555734031
Fallow/Idle Cropland 0.529354378 0.359912203 0.529279272 0.529354378 0.529316822
Cotton 0.452972025 0.32580719 0.537155554 0.452972025 0.491485025
Sorghum 0.615285898 0.39095417 0.517441972 0.615285898 0.562138104
Other 0.458951429 0.326831026 0.531687166 0.458951429 0.492649056

contribution: @hanxLi @samKhallaghi

robmarkcole commented 1 year ago

For ref the scores are also on https://huggingface.co/ibm-nasa-geospatial/Prithvi-100M-multi-temporal-crop-classification - however the above table is more complete, can you point to the source?