import_uow_radiosonde_data error - zero humidity 0-1 km

[csarantos-ceres]

Hello, I stumbled upon what I think is an issue with the U of Wyoming Radiosonde import helper function SixSHelpers.radiosonde.import_uow_radiosonde_data, where, due to an issue with some of U of W's data, the absolute humidity ends up being set to zero for the first atmospheric "slice" from 0 to 1 km.

This stems from an issue in U of Wyoming's interpolation of the raw Radiosonde data that sometimes (and for periods of months at a time) generates a partially empty row at the top of their text file. See for instance: this URL for Lincoln, IL, USA, 2019-10-31 00:00Z Where the first few lines are:

`----------------------------------------------------------------------------- PRES HGHT TEMP DWPT RELH MIXR DRCT SKNT THTA THTE THTV hPa m C C % g/kg deg knot K K K

1000.0 143
996.0 178 3.6 3.0 96 4.79 10 8 277.1 290.3 277.9 992.0 211 3.4 2.7 95 4.71 11 9 277.2 290.2 278.0`

The raw IGRA radiosonde data does have valid data starting at the very first point, so that partly empty line must be coming from U of W's interpolation? This is the header and first line of the raw radiosonde data for that day: #USM00074560 2018 11 15 00 2302 189 ncdc-nws ncdc-nws 401517 -893383 21 0 100826B 179 -20B 750 38 54 21

Now, when import_uow_radiosonde_data calls this on L323: array = np.genfromtxt(s,skip_header=4, delimiter=7,usecols=(0, 1, 2, 5),filling_values=0) ... mixing_ratio = array[:, 3]

mixing_ratio[0] (and all other missing parameters) is set to zero.

Then when _import_from_arrays interpolates the zero altitude point from this: f_interp_mixrat = interp1d(altitude, mixing_ratio, bounds_error=False, fill_value=mixing_ratio[0]) We end up with zero absolute humidity at zero altitude, only picking it up again at 1 km. I haven't checked the total precipitable water integration that happens next, but I'm guessing this zero humidity point will have a fairly large effect on that?

The intent here is good, to set the zero altitude point to the lowest altitude radiosonde point (often ~100m), but that partially empty row in the U of W data ends up setting mixing_ratio[0] to zero and thus also zeroes the humidity of the zero altitude point.

I don't know how widespread this U of W issue is, but it covers at least several months of 2019 historical data for this particular station (it appears to have gone away 15 days ago?).

I would propose a fix along the following lines, checking if the first U of W data row has an invalid number of columns and skip past it if so:

    table = "\n".join(spl)

    # Check for partly empty first line in U of W data which impacts interpolations:
    if len(table.split('\n')[4].split()) != 11:
      num_skip = 5
    else:
      num_skip = 4

    # Import to NumPy arrays
    s = io.BytesIO(table.encode())
    array = np.genfromtxt(s,skip_header=num_skip, delimiter=7,usecols=(0, 1, 2, 5),filling_values=0)

A more general solution might check the first few rows, but so far I've only seen this issue in the first row.

[robintw]

Thanks for reporting this. I agree, the code isn't dealing with the incorrect UoW data properly.

I'll try and get to dealing with this as soon as I can, but I can't guarantee how soon it will be. It's definitely on my to-do list though!

[csarantos-ceres]

Thanks, Robin for adding this to the list! I've addressed it locally for now. So no worries if it takes some time to get to. I'm not sure how many people are using this radiosonde import feature and also querying the specific IGRA sites & times that are impacted, but just in case, it's at least documented now.

[csarantos-ceres]

Thanks!