openeemeter / eeweather

Fetch NCDC ISD, TMY3, or CZ2010 weather data that corresponds to ZIP Code Tabulation Areas or Latitude/Longitude.
http://eeweather.openee.io/
Apache License 2.0
50 stars 19 forks source link

select_station breaks when some metadata values in rank_stations result are NaNs #29

Closed enovity closed 6 years ago

enovity commented 6 years ago

The select_station function tries to create an ISDStationinstance for each weather station in the rank_stations result. When creating each ISDStation instance, the instance creation attempts to call the self._load_medata function, which attempts to caste numerical metadata values as floats. When they're missing from the metadata database, it causes a type error.

Example traceback:

Traceback (most recent call last):
  File "/.../controllers.py", line 280, in fetch_weather_data
    station, warnings = eeweather.select_station(ranked_stations, coverage_range=(start_date_utc, end_date_utc))
  File "/.../test_env/lib/python3.6/site-packages/eeweather/ranking.py", line 365, in select_station
    station = ISDStation(usaf_id)
  File "/.../test_env/lib/python3.6/site-packages/eeweather/stations.py", line 1013, in __init__
    self._load_metadata()
  File "/.../test_env/lib/python3.6/site-packages/eeweather/stations.py", line 1043, in _load_metadata
    self.elevation = float(metadata['elevation'])  # meters
TypeError: float() argument must be a string or a number, not 'NoneType'
philngo commented 6 years ago

Do you happen to know the usaf_id for the station it's failing on? Definitely a bug, I'd like to reproduce it locally.

enovity commented 6 years ago

@philngo Not sure exactly, but if you just go into the metadata database and grab one of the weather stations where the elevation is null and try to instantiate an ISDStation, I think it will trigger the error. I replaced all the null elevations with a very high value as a hacky workaround, so this doesn't work on my machine now.

from eeweather.connections import metadata_db_connection_proxy
from eeweather import ISDStation

conn = metadata_db_connection_proxy.get_connection()
cur = conn.cursor()
cur.execute('''
                   select
                      usaf_id
                   from
                      isd_station_metadata
                   where
                      elevation IS NULL
                ''')
row = cur.fetchone()
station = ISDStation(row[0])
philngo commented 6 years ago

@enovity I found one - should be fixed now, released as 0.3.6.