Open BaptisteVandecrux opened 2 months ago
For info, this is due to the fact that the last transmission with good *_i
data is more than 91 days old.
Code crashes when there are recent coordinates but no recent instantaneous values.
This issue will be fixed:
SDM is fixed but LYN_T has the same issue:
2024-06-25 11:11:53,719; INFO; pypromice.postprocess.get_bufr; ####### Processing LYN_T #######
2024-06-25 11:11:53,719; INFO; pypromice.postprocess.get_bufr; Generating /mnt/data/aws/pypromice_aws/aws-bufr/BUFR_out/LYN_T.bufr from /mnt/data/aws/pypromice_aws/aws-l2/tx/LYN_T/LYN_T_hour.csv
2024-06-25 11:11:53,971; INFO; pypromice.postprocess.real_time_utilities; TIMESTAMP: 2023-12-14 00:00:00
2024-06-25 11:11:53,971; INFO; pypromice.postprocess.real_time_utilities; finding positions
2024-06-25 11:11:53,973; INFO; pypromice.postprocess.real_time_utilities; last transmission: 2024-04-19 11:00:00
2024-06-25 11:11:54,003; ERROR; pypromice.postprocess.get_bufr; Failed processing LYN_T
Traceback (most recent call last):
File "pandas/_libs/index.pyx", line 548, in pandas._libs.index.DatetimeEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 2263, in pandas._libs.hashtable.Int64HashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 2273, in pandas._libs.hashtable.Int64HashTable.get_item
KeyError: 1702512000000000000
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/aws/miniconda3/envs/py38/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3803, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 516, in pandas._libs.index.DatetimeEngine.get_loc
File "pandas/_libs/index.pyx", line 550, in pandas._libs.index.DatetimeEngine.get_loc
KeyError: Timestamp('2023-12-14 00:00:00')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/aws/miniconda3/envs/py38/lib/python3.8/site-packages/pandas/core/indexes/datetimes.py", line 736, in get_loc
return Index.get_loc(self, key, method, tolerance)
File "/home/aws/miniconda3/envs/py38/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3805, in get_loc
raise KeyError(key) from err
KeyError: Timestamp('2023-12-14 00:00:00')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/aws/miniconda3/envs/py38/lib/python3.8/site-packages/pypromice/postprocess/get_bufr.py", line 388, in get_bufr
station_position = process_station(
File "/home/aws/miniconda3/envs/py38/lib/python3.8/site-packages/pypromice/postprocess/get_bufr.py", line 209, in process_station
latest_data = get_latest_data(
File "/home/aws/miniconda3/envs/py38/lib/python3.8/site-packages/pypromice/postprocess/real_time_utilities.py", line 74, in get_latest_data
s_current = df_limited.loc[last_valid_index]
File "/home/aws/miniconda3/envs/py38/lib/python3.8/site-packages/pandas/core/indexing.py", line 1073, in __getitem__
return self._getitem_axis(maybe_callable, axis=axis)
File "/home/aws/miniconda3/envs/py38/lib/python3.8/site-packages/pandas/core/indexing.py", line 1312, in _getitem_axis
return self._get_label(key, axis=axis)
File "/home/aws/miniconda3/envs/py38/lib/python3.8/site-packages/pandas/core/indexing.py", line 1260, in _get_label
return self.obj.xs(label, axis=axis)
File "/home/aws/miniconda3/envs/py38/lib/python3.8/site-packages/pandas/core/generic.py", line 4056, in xs
loc = index.get_loc(key)
File "/home/aws/miniconda3/envs/py38/lib/python3.8/site-packages/pandas/core/indexes/datetimes.py", line 738, in get_loc
raise KeyError(orig_key) from err
KeyError: Timestamp('2023-12-14 00:00:00')
I have copied the logs to
process_stdout_err_SDM
andprocess_stderr_err_SDM
@PennyHow , @ladsmund could you have a look at it?