tjturnage / cloud-radar-server

Radar server in the cloud for NOAA computing project
MIT License
2 stars 2 forks source link

hodo_plot.py does not work for older nexrad files #25

Closed lcarlaw closed 1 month ago

lcarlaw commented 2 months ago

It seems probable this is due to file extensions like .gz in some older files (see 2013-05-20 19-20z from TLX for example). A possible workaround could be to use the glob module or a regular expression to try to match radar file names. From the pyart docs, pyart.io.read() looks like it should be able to handle .gz files.

scottthomaswx commented 2 months ago

I was able to successfully use the aforementioned case/gz files with no issue in my code. Unsure if the failure lies in the refactored version in the server that TJ edited. Is there some sort of logging that shows what line the issue resides in?

lcarlaw commented 2 months ago

There is actually no error, and that's because no radars are found in the case of the older .gz files. I think the radar_files and radar_filepaths lists are all empty, so the main loop is never entered and the script exits silently.

Within the /data/radar/KTLX/downloads directory (for the 5/20/2013 case), the original files end with .gz. There are also .uncompressed files, as well as files that end with no filename (these are the gunzipped files and are exactly the same as the .uncompressed files). I think to correct, the logic to look for radar_files and their paths just needs to look for files that either end with .gz, or VXX and that should solve the problem. The .endswith seems to work well, but it's also possible to use the glob module I think.

from glob import glob 
files = []
endings = ('*.gz', '*V06', '*V08')
for end in endings:
    files.extend(glob(f"{DOWNLOADS}/{end}"))

or something. I think searching for those file endings could work?

lcarlaw commented 1 month ago

Fully addressed this read issue with #34 above.