Closed douglasdennis closed 1 year ago
Looks like #138 broke the test_fetch_historical_with_consistent_columns test for the AWSWeatherProvider.
test_fetch_historical_with_consistent_columns
================================================================================== FAILURES ================================================================================== _______________________________________________________________ test_fetch_historical_with_consistent_columns ________________________________________________________________ weather_provider = <rlf.forecasting.data_fetching_utilities.weather_provider.aws_weather_provider.AWSWeatherProvider object at 0x7eff80d33460> @pytest.mark.aws @pytest.mark.slow def test_fetch_historical_with_consistent_columns(weather_provider): expected_columns = [ 'soil_moisture_level_1', 'soil_moisture_level_2', 'soil_moisture_level_3', 'soil_moisture_level_4', 'soil_temperature_level_1', 'soil_temperature_level_2', 'soil_temperature_level_3', 'soil_temperature_level_4', ] requested_columns = [ 'soil_moisture_level_1', 'soil_moisture_level_2', 'soil_moisture_level_3', 'soil_moisture_level_4', 'soil_temperature_level_1', 'soil_temperature_level_2', 'soil_temperature_level_3', 'soil_temperature_level_4', ] weather_provider.set_timestamp(CURRENT_TESTING_TIMESTAMP) > weather_datums = weather_provider.fetch_historical(columns=requested_columns) tests/forecasting/data_fetching_utilities/weather_provider/test_aws_weather_provider.py:178: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ src/rlf/forecasting/data_fetching_utilities/weather_provider/aws_weather_provider.py:80: in fetch_historical datums = self.download_datums_from_aws(dir_path="historical", columns=columns) src/rlf/forecasting/data_fetching_utilities/weather_provider/aws_weather_provider.py:53: in download_datums_from_aws datum = self.aws_dispatcher.download_datum( src/rlf/aws_dispatcher.py:144: in download_datum hourly_parameters = self.download_df_from_parquet(folder_name, "data", columns=columns) src/rlf/aws_dispatcher.py:102: in download_df_from_parquet table = dataset.read(columns=columns) venv/lib/python3.10/site-packages/pyarrow/parquet/core.py:2601: in read table = self._dataset.to_table( pyarrow/_dataset.pyx:369: in pyarrow._dataset.Dataset.to_table ??? pyarrow/_dataset.pyx:336: in pyarrow._dataset.Dataset.scanner ??? pyarrow/_dataset.pyx:2576: in pyarrow._dataset.Scanner.from_dataset ??? pyarrow/_dataset.pyx:2484: in pyarrow._dataset.Scanner._make_scan_options ??? pyarrow/_dataset.pyx:2385: in pyarrow._dataset._populate_builder ??? _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > ??? E pyarrow.lib.ArrowInvalid: No match for FieldRef.Name(soil_moisture_0_to_7cm) in temperature_2m: double E relativehumidity_2m: int64 E dewpoint_2m: double E apparent_temperature: double E pressure_msl: double E surface_pressure: double E precipitation: double E rain: double E snowfall: double E cloudcover: int64 E cloudcover_low: int64 E cloudcover_mid: int64 E cloudcover_high: int64 E shortwave_radiation: double E direct_radiation: double E diffuse_radiation: double E direct_normal_irradiance: double E windspeed_10m: double E windspeed_100m: double E winddirection_10m: int64 E winddirection_100m: int64 E windgusts_10m: double E et0_fao_evapotranspiration: double E vapor_pressure_deficit: double E soil_temperature_level_1: double E soil_temperature_level_2: double E soil_temperature_level_3: double E soil_temperature_level_4: double E soil_moisture_level_1: double E soil_moisture_level_2: double E soil_moisture_level_3: double E soil_moisture_level_4: double E time: timestamp[us, tz=UTC] E __fragment_index: int32 E __batch_index: int32 E __last_in_fragment: bool E __filename: string pyarrow/error.pxi:100: ArrowInvalid ========================================================================== short test summary info =========================================================================== FAILED tests/forecasting/data_fetching_utilities/weather_provider/test_aws_weather_provider.py::test_fetch_historical_with_consistent_columns - pyarrow.lib.ArrowInvalid: No match for FieldRef.Name(soil_moisture_0_to_7cm) in temperature_2m: double
Looks like #138 broke the
test_fetch_historical_with_consistent_columns
test for the AWSWeatherProvider.