GOES should support a different AWS bucket or data access method in general

[astrobokonon]

The one I started with is flaking out more, not sure why but probably the govt. shutdown isn't helping. I think there's a Unidata source on AWS as well as a data distribution service that uses a different app of some sort, but my ideal would be to switch over to the Unidata AWS source to see if it's more stable.

https://registry.opendata.aws/unidata-goes/ has a slightly different access scheme, I think they're basically just a flat dump of the most recent X days worth of data in goesingest/ rather than nested directories by date like the NOAA one.

[astrobokonon]

Looking into it today, the Unidata AWS bucket appears to not have any L2 data in it. Bummer.

awsbucket = 'goesingest'
awszone = 'us-west-2'
s3 = boto3.resource('s3', awszone,
                    aws_access_key_id=aws_keyid,
                    aws_secret_access_key=aws_secretkey)
buck = s3.Bucket(awsbucket)
qt = "OR_ABI-L2"
curdata = buck.objects.filter(Prefix=qt)
for o in curdata:
    print(o.key)

returned no results at all. Using a chrome extension to manually browse the bucket I see the same thing, so it looks like I'd have to consider falling back to raw radiance data products (ugh) or just advertising to the end user that there are the potential for outages beyond our control.

An even fancier scenario could be to replace G16 with G17 data if there is an outage detected; https://registry.opendata.aws/noaa-goes/ has a 'noaa-goes17' bucket that's now available, and all of AZ is still captured in that view. But G17 data are at an even earlier life stage than G16 so that doesn't make much sense to try to do, really.

[astrobokonon]

While looking at something else, this makes me think that maybe switching to GOES-17 would be better once it actually becomes official GOES-West would be better; pixel scales are better for AZ there:

Just to keep this in mind. It really doesn't matter all that much for our use case though.

[astrobokonon]

...But, if the CONUS sector really is as shown, the lower SE corner of AZ might be cut off. If that's the case, we'd have to download the full disk images and that'd be a bigger task in general since the transformation would take longer to calculate and the images would be much larger to download. Probably not worth it?

[astrobokonon]

Thinking about this more, this is really a stretch for where we're at currently. Once GOES-17 is officially designated as GOES-W, it's worth considering; but that time frame is still somewhere beyond July 2019, at which point NightWatch should have been up and running for a while and we'll have a better idea on what is actually needed to improve functionality.