search

LowellObservatory / NightShift

Collection of sub-modules used in the NightWatch (and other Night*) project
Mozilla Public License 2.0
1 stars 1 forks source link

Unhandled exception when processing bad radar data #1

Closed astrobokonon closed 5 years ago

astrobokonon commented 5 years ago

Noticed this tonight:

2019-05-27 06:20:31,666 INFO     Reading: ./outputs//raws/KFSX20190526_211536
2019-05-27 06:20:34,405 ERROR    /opt/conda/lib/python3.7/site-packages/pyart/io/nexrad_level2.py:187: UserWarning: No MSG5 detected. Setting to meaningless data. Rethink your life choices and be ready for errors.Specifically fixed angle data will be missing
  warnings.warn("No MSG5 detected. Setting to meaningless data. "
2019-05-27 06:20:35,182 INFO     Done reading!
2019-05-27 06:20:35,182 ERROR    Traceback (most recent call last):
2019-05-27 06:20:35,182 ERROR      File "nexradaws_looper.py", line 232, in <module>
2019-05-27 06:20:35,182 ERROR    main(outdir, creds, forceDown=forceDownloads, forceRegen=forceRegenPlot)
2019-05-27 06:20:35,182 ERROR      File "nexradaws_looper.py", line 164, in main
2019-05-27 06:20:35,183 ERROR    forceRegen=forceRegen)
2019-05-27 06:20:35,183 ERROR      File "/home/lig/Codes/RadarLove/plotNEXRAD.py", line 416, in makePlots
2019-05-27 06:20:35,183 ERROR    vcpmode = radar.metadata['vcp_pattern']
2019-05-27 06:20:35,183 ERROR    KeyError
2019-05-27 06:20:35,183 ERROR    :
2019-05-27 06:20:35,183 ERROR    'vcp_pattern'

Looks like a bad file that should have been skipped. It's not an empty file, though:

KFSX20190526_211536                                26-May-2019 21:28             4088860
astrobokonon commented 5 years ago

I quickly hacked in a fix in the old repository but it should be revisited ASAP when NightShift takes over in the testing environment.

astrobokonon commented 5 years ago

Also need to remember that the radar plotting is using the 'staticfile' scheme, but GOES is not yet on that path - that means that the HAniS setup will need to change for GOES as well.

https://github.com/LowellObservatory/NightShift/blob/59da7c02ab908f2a6c55733c765a4a31abe75283/nightshift/radar/nexradaws_looper.py#L48-L55

astrobokonon commented 5 years ago

I tried to go back and pull the offending file manually off of AWS, but 20190526 has been wiped past KFSX20190526_124701_V06. I can't find maintenance logs to see if there was an actual site problem, so this was likely a case of the radar producing bad data that was cleared from the archive.

If anything, it might be worth going back and trying to figure out how to actually raise/catch the warning from reading:

No MSG5 detected. Setting to meaningless data.

but what I've got works for now, so this is a bit moot. Closing.