Read NEXRAD Level 2 files without Message 31

[seberry2]

I am trying to work set of NEXRAD level 2 files where some contain Message 31, but a significant portion do not. In the current version of Py-ART, I see that only the message 31 files are readable. Have there been any recent updates or developments that will make the message 1 files readable? If not, how far off is this feature?

[jjhelmus]

Message 1 files are (still) not supported naively by Py-ART but I would like to see support for these files added. I cannot give a firm date on when this will be added but likely a long while off. If I can convince the ARM program that there is a need for this it will become a higher priority. Others are also add such support to Py-ART.

For the time being you can convert Level 2 NEXRAD files in message 1 format with the Unidata toolsUI program to netCDF files. The pyart.io.read_nexrad_cdm function in Py-ART should be able to read these files. I also believe RadX supports message 1 format and can convert into a Cf/Radial which Py-ART can read.

[jjhelmus]

I did some work on reading NEXRAD Level 2 files which use Message 1 records to store the radial data. Adding support for reading the data from this to the NEXRADLevel2File class is relatively straightforward but how this data should be represented in a Radar object requires some though.

The issue is that message 1 files contain three fields, reflectivity, Doppler velocity, and spectral width. The reflectivity field uses a 1000 meter gate spacing with a maximum range of ~450 km, where as the velocity and spectral width fields are on a 250 meter spacing with maximum range of ~230 km. Py-ART's Radar class requires that all field data have the same range, so the raw data from all three fields cannot be stored in a single Radar instance. A few possible solutions to this problem that come to mind:

• Return two Radar instances when reading message 1 files. The first contains the reflectivity data on a 1000 meter spacing and the second the velocity and spectral width data on a 250 meter gate spacing.
• Interpolate or reduce one set of fields to match the gate resolution of the other. This is is procedure that Radx uses when converting message 1 files to CfRadial, the reflectivity field is interpolated upon a ray with 250 meter gate spacing.

The former option provides a more realistic view of the original data to the user but would have the read_nexrad_level2 behavior differently than other read_ function in that a tuple is returned. Additionally using these two radar objects does not fit well with the current Py-ART workflow where a single Radar object is passed to the various processing and visualization routines. The latter option keeps a uniform nature across the read_ functions which better fits into the current Py-ART workflow at the expensive of a of mis-representing the data.

Personally I think both options should be available with the default being a single Radar instance being returned where the reflectivity is interpolated.

[tjlang]

I would vote for option 2, since Py-ART should be compatible with Radx given it can call Radx. I could see potential issues where using Py-ART's Radx read routine gives one radar object, but an updated NEXRAD read gives a different type of radar object, for the same file.

Also, I've been using Radx to bypass this problem and just convert the NEXRAD away from Level 2 so Py-ART can read the data.

So if you do one option, do the second. If you do both, make option 2 the default.

[jjhelmus]

Should mention a third option: Represent both sets of fields on a single range dimension which contain the locations of both the 250m and 1000m resolutionss. Gate locations where a paricular field was not collected would be masked.

Downside of this method would be a larger sized Radar object, a non-uniform range spacing (at least in the case I choose the velocities range start at -375 m, reflectivities at 0 m, so the gate locations are offset by 125m), and ambiguity in the interpretation of a masked gates as to it represent a gate which was not collected or below a detection level (although NEXRAD data in Py-ART already contains some of these ambiguities).

[jjhelmus]

Support for reading NEXRAD Level 2 files contain message 1 records was added to Py-ART with PR #455. By default, data in low resolution scans are interpolated to higher resolution when the volume contains both high and low resolution scans. When this interpolation is performed a warning message is raised informing the users of this action. A Radar instance containing the original, non-interpolated data can be obtained by selecting out the low resolution scans and fields using the scans and exclude_fields parameters.