Unidata / siphon

Siphon - A collection of Python utilities for retrieving atmospheric and oceanic data from remote sources, focusing on being able to retrieve data from Unidata data technologies, such as the THREDDS data server.
https://unidata.github.io/siphon
BSD 3-Clause "New" or "Revised" License
215 stars 75 forks source link

Exception handling needed in _get_data method of WyomingUpperAir for missing or erroneous metadata #751

Open nerojia opened 10 months ago

nerojia commented 10 months ago

Body:

Hello,

I encountered an issue while using the WyomingUpperAir class to download and parse upper air observations, specifically when dealing with data from certain stations that have missing or erroneous metadata.

For example, the entire 2022 data for station 95282 has missing or incorrect values for Station latitude , Station elevation and Station longitute. The current implementation of the _get_data method does not handle these cases well, leading to a ValueError when it tries to parse these fields into floats.

Here is a sample traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: could not convert string to float: '******'

This issue could be resolved by adding exception handling to the float conversion process. Here's a proposed solution:

# Safe parsing of float values
def safe_float(value):
    try:
        return float(value)
    except ValueError:
        return np.nan

latitude = safe_float(lines[4].split(':')[1].strip())
longitude = safe_float(lines[5].split(':')[1].strip())
elevation = safe_float(lines[6].split(':')[1].strip())
pw = safe_float(lines[-1].split(':')[1].strip())

In this solution, a helper function safe_float is used to attempt the float conversion, and if it fails, np.nan is returned instead. This prevents a ValueError from being raised and allows the function to complete successfully, even if some metadata fields cannot be converted to floats.

This issue is particularly important because it affects the usability of the library for any station with missing or erroneous metadata, not just station 95282.

I hope this information is helpful and look forward to your feedback.

Best regards, Nero Jia

tkschuler commented 10 months ago

Duplicate of #749

nerojia commented 10 months ago

Duplicate of #749

Thank you for your response! I understand, and I appreciate it!