search

M3Works / metloom

Location Oriented Observed Meteorology
Other
15 stars 4 forks source link

Added more logic arounce CDEC parse_sensor_table #83

Closed micahjohnson150 closed 1 year ago

micahjohnson150 commented 1 year ago

Exception was being thrown when attempting to be parse a table whose number of columns in the differed from what is in _parse_sensor_table(self, df). This particular case happens when we attempt to parse metadata for GDV. This originally attempts to parse the Zero Datum table as the Sensor Description table.

To replicate try: 

    pnt = CDECPointData('GDV', 'EAST FORK CARSON R')
    print(pnt.metadata)

Fails on https://github.com/M3Works/metloom/blob/aa83b058bf9432c42c432c5d933d9bcc93baa1ff/metloom/pointdata/cdec.py#L76

Exception

  File "/home/micah/projects/m3works/metloom/metloom/pointdata/cdec.py", line 76, in _parse_meta_page
    df_sensors = self._parse_sensor_table(df[1])
  File "/home/micah/projects/m3works/metloom/metloom/pointdata/cdec.py", line 40, in _parse_sensor_table
    df_sensors = pd.DataFrame(
  File "/home/micah/projects/venv/wranglerenv/lib/python3.8/site-packages/pandas-1.3.4-py3.8-linux-x86_64.egg/pandas/core/frame.py", line 672, in __init__
    mgr = ndarray_to_mgr(
  File "/home/micah/projects/venv/wranglerenv/lib/python3.8/site-packages/pandas-1.3.4-py3.8-linux-x86_64.egg/pandas/core/internals/construction.py", line 324, in ndarray_to_mgr
    _check_values_indices_shape_match(values, index, columns)
  File "/home/micah/projects/venv/wranglerenv/lib/python3.8/site-packages/pandas-1.3.4-py3.8-linux-x86_64.egg/pandas/core/internals/construction.py", line 393, in _check_values_indices_shape_match
    raise ValueError(f"Shape of passed values is {passed}, indices imply {implied}")
ValueError: Shape of passed values is (1, 4), indices imply (1, 6)

Fix: The new logic enforces the columns to match before attempting to parse.