Replace ``download_noaa_aws.sh`` with a pythonic implementation which does the same thing

[evrose54]

Requested Update

Description

download_noaa_aws.sh uses rclone which we view as an unnecessary system dependency that could be avoided by creating a pythonic version of that file which does the same thing. Hopefully, we can create this module within the next release so we have a manageable way of tracking / installing test datasets used for GeoIPS. This could be done via http requests and dumping that data to your machine with GeoIPS installed, or a different implementation. We can discuss exactly how we want to do this later on.

Background and Motivation

This came up as a conversation between @jsolbrig and myself when modifying download_noaa_aws.sh to get geokompsat test data.

Alternative Solutions

Leave the code as is and deal with it.

Checklist for Completion

• [ ] Discuss a pythonic solution that will accomplish this
• [ ] Implement it.

[evrose54]

It seems that download_noaa_aws.sh is no longer used within GeoIPS source code. Per a conversation with @jsolbrig, we've decided it's probably best to separate data collection routines into a standalone package which will do just that. Leaving this comment on this issue so we can address this at a later point.

[evrose54]

These steps should be addressed when moving the test_data setup over to a new package.

• [ ] New repo for test datasets.
• [ ] Collect datasets from nrl ftp and noaa aws and GitHub
• [ ] Put them on cira next cloud instance with applicable names - sensor separated
• [ ] Move all of the scripts over
• [ ] Remove all git related info from tar files
• [ ] keep download_noaa_aws in geoips. Just copy it across to this package
• [ ] consider moving get_test_datasets from readers to unit tests

[evrose54]

If you have access see https://github.com/NRLMMD-GEOIPS/cira_test_data_maintenance

[jsolbrig]

This is complete in NRLMMD-GEOIPS/cira_test_data_maintenance