owid / etl

A compute graph for loading and transforming OWID's data
https://docs.owid.io/projects/etl
MIT License
78 stars 21 forks source link

📊 Update Near-surface temperature anomaly data #2925

Closed pabloarosado closed 3 months ago

owidbot commented 3 months ago
Quick links (staging server): Site Admin Wizard

Login: ssh owid@staging-site-update-temperature-anomaly

chart-diff: ✅
  • 1/1 reviewed charts
    • Modified: 1/1
    • New: 0/0
data-diff: ```diff + Dataset garden/met_office_hadley_centre/2024-07-02/near_surface_temperature + + Table near_surface_temperature + + Column temperature_anomaly + + Column lower_limit + + Column upper_limit 2024-07-03 11:04:52 [error ] Traceback (most recent call last): File "/home/owid/etl/.venv/lib/python3.10/site-packages/requests/models.py", line 974, in json return complexjson.loads(self.text, **kwargs) File "/home/owid/etl/.venv/lib/python3.10/site-packages/simplejson/__init__.py", line 514, in loads return _default_decoder.decode(s) File "/home/owid/etl/.venv/lib/python3.10/site-packages/simplejson/decoder.py", line 386, in decode obj, end = self.raw_decode(s) File "/home/owid/etl/.venv/lib/python3.10/site-packages/simplejson/decoder.py", line 416, in raw_decode return self.scan_once(s, idx=_w(s, idx).end()) simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/owid/etl/etl/datadiff.py", line 423, in cli lines = future.result() File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result return self.__get_result() File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result raise self._exception File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/home/owid/etl/etl/datadiff.py", line 416, in func differ.summary() File "/home/owid/etl/etl/datadiff.py", line 254, in summary self._diff_tables(self.ds_a, self.ds_b, table_name) File "/home/owid/etl/etl/datadiff.py", line 122, in _diff_tables table_a = future_a.result() File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result return self.__get_result() File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result raise self._exception File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/home/owid/etl/.venv/lib/python3.10/site-packages/tenacity/__init__.py", line 330, in wrapped_f return self(f, *args, **kw) File "/home/owid/etl/.venv/lib/python3.10/site-packages/tenacity/__init__.py", line 467, in __call__ do = self.iter(retry_state=retry_state) File "/home/owid/etl/.venv/lib/python3.10/site-packages/tenacity/__init__.py", line 368, in iter result = action(retry_state) File "/home/owid/etl/.venv/lib/python3.10/site-packages/tenacity/__init__.py", line 390, in self._add_action_func(lambda rs: rs.outcome.result()) File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result return self.__get_result() File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result raise self._exception File "/home/owid/etl/.venv/lib/python3.10/site-packages/tenacity/__init__.py", line 470, in __call__ result = fn(*args, **kwargs) File "/home/owid/etl/etl/datadiff.py", line 837, in get_table_with_retry return ds[table_name] File "/home/owid/etl/etl/datadiff.py", line 278, in __getitem__ return tables.load() File "/home/owid/etl/lib/catalog/owid/catalog/catalogs.py", line 312, in load return self.iloc[0].load() # type: ignore File "/home/owid/etl/lib/catalog/owid/catalog/catalogs.py", line 363, in load return Table.read(uri) File "/home/owid/etl/lib/catalog/owid/catalog/tables.py", line 177, in read table = cls.read_feather(path) File "/home/owid/etl/lib/catalog/owid/catalog/tables.py", line 360, in read_feather cls._add_metadata(df, path) File "/home/owid/etl/lib/catalog/owid/catalog/tables.py", line 333, in _add_metadata metadata = cls._read_metadata(path) File "/home/owid/etl/lib/catalog/owid/catalog/tables.py", line 394, in _read_metadata return cast(Dict[str, Any], requests.get(metadata_path).json()) File "/home/owid/etl/.venv/lib/python3.10/site-packages/requests/models.py", line 978, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) âš  Found errors, create an issue please Legend: +New ~Modified -Removed =Identical Details Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippet ``` Automatically updated datasets matching _weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk_ are not included

Edited: 2024-07-03 11:04:55 UTC Execution time: 65.51 seconds