This PR introduces a number of significant updates:
[x] Model updates: Presto previously considered SRTM as dynamic in time (num_timesteps SRTM tokens were passed through the model), even though it was only collected at a single timestep. A single SRTM token is now passed through the model. In addition, a bug in the decoder which shuffled some bands during reconstruction has been fixed.
[x] Masking updates: The masking functions were updated to work with both dynamic-in-time and static-in-time inputs. In addition, they were updated to accept arbitrary masking ratios (previously, the masking ratio had to yield a number of masked tokens divisible by the number of timesteps and the number of band groups).
The default_model.pt weights have now been updated with a model trained after these changes.
[x] Evaluation updates (images): For evaluation tasks which make a single prediction for an image, we now pass a mean and standard deviation of the pixel-outputs for that image to a downstream classifier (instead of passing each individual pixel to the classifier and taking the mode of the classifier's outputs).
[x] Evaluation updates (seeds): All downstream classifiers with randomness (i.e. everything except the K-nearest-neighbours classifier and the linear regression) are run for 3 seeds to measure Presto's sensitivity to seeds.
[x] Evaluation updates (EuroSat): Our EuroSat evaluation now better mirrors the approach taken by previous works; we downsample images to different resolutions, and process all the pixels in the image. In addition, we use the splits described #8).
A manuscript with updated results will be posted on arxiv The latest arxiv version of the paper reflects these changes.
[x] Many more tests: We add many more tests to ensure our masking functions are doing what we expect, and to ensure tokens are being handled in the right way by the Presto encoder and decoder.
[x] Other infrastructure updates: Lots of changes here: (i) we add logging to the training and evaluation code for clarity, (ii) speed up evaluation tasks by using multiple workers for the eval dataloaders, (iii) replace the processing code for TreeSat and EuroSat with a dataloader which reads the raw files (so there is no need to process the data before running th evaluation code), (iv) run CropHarvest validation less often to speed up pre-training, (iv) make the data_dir configurable in train.py, (vi) dump the evaluation results in a json file locally in addition to (optionally) storing them on wandb.
This PR introduces a number of significant updates:
[x] Model updates: Presto previously considered SRTM as dynamic in time (
num_timestepsSRTM tokens were passed through the model), even though it was only collected at a single timestep. A single SRTM token is now passed through the model. In addition, a bug in the decoder which shuffled some bands during reconstruction has been fixed.[x] Masking updates: The masking functions were updated to work with both dynamic-in-time and static-in-time inputs. In addition, they were updated to accept arbitrary masking ratios (previously, the masking ratio had to yield a number of masked tokens divisible by the number of timesteps and the number of band groups).
The
default_model.ptweights have now been updated with a model trained after these changes.[x] Evaluation updates (images): For evaluation tasks which make a single prediction for an image, we now pass a mean and standard deviation of the pixel-outputs for that image to a downstream classifier (instead of passing each individual pixel to the classifier and taking the mode of the classifier's outputs).
[x] Evaluation updates (seeds): All downstream classifiers with randomness (i.e. everything except the K-nearest-neighbours classifier and the linear regression) are run for 3 seeds to measure Presto's sensitivity to seeds.
[x] Evaluation updates (EuroSat): Our EuroSat evaluation now better mirrors the approach taken by previous works; we downsample images to different resolutions, and process all the pixels in the image. In addition, we use the splits described #8).
A manuscript with updated results will be posted on arxivThe latest arxiv version of the paper reflects these changes.data_dirconfigurable intrain.py, (vi) dump the evaluation results in a json file locally in addition to (optionally) storing them on wandb.The repository before this PR is tagged
v0.1.