Refactor NOAA20 and SUOMI Downloads Should Use FIRMS API with MAP_KEY

[ranchodeluxe]

Problem

We don't want to continue refreshing tokens with the existing ED downloads. FIRMS has an API we can use with non-expiring tokens

Solution (from Otmar's emails)

You can use our API and avoid dealing with the token. You just need to put your email and it will send you the key. https://firms.modaps.eosdis.nasa.gov/api/map_key/

Once you have it, use: https://firms.modaps.eosdis.nasa.gov/api/area/

replace MAP_KEY with the one sent to you and then you can query the WORLD for TODAY as: https://firms.modaps.eosdis.nasa.gov/api/area/csv/MAP_KEY/VIIRS_NOAA20_NRT/world/1 https://firms.modaps.eosdis.nasa.gov/api/area/csv/MAP_KEY/VIIRS_NOAA21_NRT/world/1 Note: no recent data for SNPP right now https://firms.modaps.eosdis.nasa.gov/api/area/csv/MAP_KEY/VIIRS_SNPP_NRT/world/1

If you need the last 2 days: https://firms.modaps.eosdis.nasa.gov/api/area/csv/MAP_KEY/VIIRS_NOAA20_NRT/world/2

By default nasa.gov emails get 100,000 transactions / 10-minute window which is more than enough for most ingest scripts. (most barely use 1000). There is not expiration on our MAP_KEY and the implementation is way simpler.

However, if it is more convenient to use EarthData token, then do contact them. I think that 2-month expiration period is horrible for anyone downloading data regularly, but I guess there was a reason why free public data has to have that level of protection.

Let me know if you need any help or additional instructions.

AC

0. walk through the examples above to get an understanding
1. NRT data has it's own 7-day window to download and uses source "VIIRS_NOAA20_NRT" || "VIIRS_SNPP_NRT": https://firms.modaps.eosdis.nasa.gov/api/area/
2. archival data will the s3-hosted products on NETCDF and LPDAC (VNP14IMG, VJ114IMG) (according to Shane)
3. check for any schema changes or things that might cause problems
4. refactor existing

[ranchodeluxe]

we all agree we can do this after the merge

[mccabete]

@mccabete re-review post Shane debrief

[zebbecker]

Part of this refactor should include adding logic to gracefully handle failures from one satellite