Buffalo soundings cut off at 400 hpa for Sept 29 at 12 UTC through Siphon

[eliteuser26]

Retrieved the Buffalo soundings for September 29 at 12 UTC from the Wyoming web site through Siphon. The sounding retrieved gets cut off at 400 hpa. Was able to reproduce it on my web site and through Jupyter Notebook. When verifying the Pandas dataframe data it does cut off at that pressure level. Upon verifying with the raw data on the Wyoming web site the profile goes up to 100 hpa for that time period. Seen few other examples of Skew-T profiles for Buffalo during the summer where it gets cut off exactly at 400 hpa.

Here is the Skew-T plot for Buffalo for that date and time:

Here is the python code used to generate the sounding for Buffalo in Jupyter Notebook:

from datetime import datetime, date
import pandas as pd
import pytest
from siphon.simplewebservice.wyoming import WyomingUpperAir
from metpy.units import units
import matplotlib.pyplot as plt
import metpy.plots as plots
from metpy.testing import assert_almost_equal
import numpy as np
import metpy.calc as mpcalc
import pytz
import os
current_datetime = datetime.now()
current_year = current_datetime.year
current_month = current_datetime.month
current_day = current_datetime.day
current_hour = 12
date_value = datetime(current_year, current_month, current_day,
                      current_hour, tzinfo=pytz.UTC)
date_string = date_value.strftime('%d %B %Y %H:%M:%S UTC')
station_id = 'BUF'
station_name = 'Buffalo'
df = WyomingUpperAir.request_data(date_value, station_id)
p = df['pressure'].values * units.hPa
T = df['temperature'].values * units.degC
Td = df['dewpoint'].values * units.degC
u = df['u_wind'].values * units.knot
v = df['v_wind'].values * units.knot
fig = plt.figure(figsize=(10, 10))
skew = plots.SkewT(fig)
skew.plot(p, T, 'red')
skew.plot(p, Td, 'green')
interval = np.logspace(2, 3) * units.hPa
idx = mpcalc.resample_nn_1d(p, interval[1:])
skew.plot_barbs(p[idx], u[idx], v[idx])
if date(current_year, 5, 15):
    skew.ax.set_xlim(-50, 40)
else:
    skew.ax.set_xlim(-60, 30)
skew.plot_dry_adiabats()
skew.plot_moist_adiabats()
skew.plot_mixing_lines()
plots.add_metpy_logo(fig, x=135, y=115)
plt.title(f'{station_name} Sounding', loc='left')
plt.title(f'Valid Time: {date_string}', loc='right')
skew.ax.axvline(0, color='k')
parcel_profile = mpcalc.parcel_profile(p, T[0], Td[0])
skew.plot(p, parcel_profile, color='k')
skew.shade_cape(p, T, parcel_profile)
skew.shade_cin(p, T, parcel_profile)
plt.savefig(f'{station_id}_sounding_current.svg', bbox_inches='tight', pad_inches=0)

• Problem description: Buffalo soundings cut off at 400 hpa for Sept 29 at 12 UTC through Siphon.
• Expected output: It should plot the Buffalo profile up to 100 hpa.
• Which platform: Linux and Windows
• Versions
  • python: 3.7.3
  • Metpy: 0.12.1

[dopplershift]

Well this is fun. Siphon is returning all of the data that the site returns. However, if you change 'BUF' to '72528' you'll get all of the data--72528 is the site that is used to request data if you go through the actual web interface.

[dopplershift]

sigh For the same date (today), with station OUN/72357, I actually get more data when I ask for 'OUN' vs. 72357, so requesting the number is not a simple solution.

[dopplershift]

I sent an email to the upstream provider to seek some clarification here before we decide how to deal with this in Siphon.

[eliteuser26]

This is interesting. According to your example that I followed on the Metpy web site, you indicated to use the station letters instead of the number. I know on the Wyoming web site it actually uses the number of the station for requesting the profile. In that case I can use the number if needed. However it shouldn't make a difference whether we use the station letters or number. It should provide the same profile. I also know that the station letters for the soundings doesn't always correspond to the Metar station. Example: Detroit sounding DTX and Metar station DET. For the Buffalo station they are the same (BUF).

[dopplershift]

So based on feedback from upstream, the site shouldn't make a difference. Sounds like there's some caching at play that could be playing a role here. When you use the web form there's an option to force a re-compute which fixes it.

[eliteuser26]

Right now I use the Metpy and Siphon code to get the profile for BUF. I did a test once to regenerate the profile twice through Metpy and Siphon. It still regenerated the same profile. I will try with the number instead in the Siphon request. This is done through Python code not the Wyoming web interface.

[dopplershift]

Right, if you're only using the Siphon interface, you're going to get the same results. The option to regenerate is only currently available on the Wyoming web interface.

[eliteuser26]

Should I keep this as an issue or close it? Is this something that could be worked on? There should other options available for Metpy users as a fallback.

[dopplershift]

Leave it open for now. We could expose the "force recalculate" option. I'm just still pondering the potential consequences on the server of enabling that at a programmatic level.

[eliteuser26]

Redid the sounding for Buffalo for September 29 at 12 UTC and it worked fine now. How about the possibility of decoding raw TTAA/TTBB or decoded text file as a fallback? There was one Metpy issue related to this. Would it be the effort of creating such a function for Metpy? I could work on this if there is a need for it. Comments.

[dopplershift]

Well on the siphon side, it would be trivial to request and return the text we get for "Raw Text" or "Unmerged Text".

I glanced at using those, and one of the challenges would be merging them together (re-inventing what the U Wyoming site does), because the PPAA/PPBB give wind as a function of height, not pressure.

[eliteuser26]

I was thinking about using the text list option on Wyoming website where it contains both pressure and height in decoded format. Like you said the work has been done already by them. Would need to parse the text and save it in a Pandas dataframe. I don't know if this is feasible or not.

[dopplershift]

I don't follow. We already use "Text: List" currently for siphon. What you described is literally what Siphon is doing currently. That's the one that has the problem with the cached data cutting off BUF.

The other options that don't have the caching problem ("Text: Raw" and "Text: Unmerged"), and thus would be suitable as work-arounds, have the limitations I described above:

I glanced at using those, and one of the challenges would be merging them together (re-inventing what the U Wyoming site does), because the PPAA/PPBB give wind as a function of height, not pressure.

[eliteuser26]

I wasn't sure that is what you meant by it. So you want to start from scratch by using the raw data from what I understand. I do get it now. Sorry about that. Should have read the comments properly. You want to use the other options at Wyoming website.

[dopplershift]

I'm not necessarily proposing blowing away what we have...maybe having it as an option. Just considering the options.