search

owid / etl

A compute graph for loading and transforming OWID's data
https://docs.owid.io/projects/etl
MIT License
81 stars 22 forks source link

Chart sync CLI fails because of a `requests.exceptions.ConnectionError` #2457

Closed pabloarosado closed 7 months ago

pabloarosado commented 7 months ago

When running etl chart-sync update-faostat-data .env.live --staging-created-at 2024-03-14 I get a requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=80): Max retries exceeded with url: /admin/api/charts (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x16866b610>: Failed to establish a new connection: [Errno 61] Connection refused')).

NOTES:

  1. I'm running this script from my local computer, with a SSH tunnel to production.
  2. I temporarily replaced the assert target_chart.config["isPublished"] with a warning, because it was failing for chart 1031 (which is another issue: https://github.com/owid/etl/issues/2452 ).
  3. When running it with --dry-run it works.

Full log:

(etl-py3.11) ➜  etl git:(master) ✗ etl chart-sync update-faostat-data .env.live --staging-created-at 2024-03-14
2024-03-26 12:04:04 [info     ] staging_sync.start             chart_ids={1024, 1025, 1538, 1027, 1029, 1030, 1031, 7685, 5129, 1035, 525, 1037, 1038, 1039, 1040, 2576, 5138, 533, 534, 1046, 1559, 1562, 2086, 4652, 4653, 558, 1070, 1072, 1073, 571, 4156, 1088, 1089, 1090, 4675, 4676, 4677, 4678, 4679, 4680, 4681, 6216, 589, 590, 3676, 1127, 3175, 3176, 2673, 1149, 1154, 5624, 5116, 5625, 5626, 2713, 3738, 3236, 4774, 6550, 715, 2766, 4823, 2777, 2778, 2779, 2780, 2782, 2784, 1249, 1250, 2785, 2786, 2787, 2788, 743, 2789, 745, 746, 2790, 2791, 2792, 2793, 2794, 2795, 2796, 2797, 4848, 768, 787, 5348, 802, 823, 826, 5949, 4926, 841, 6989, 847, 848, 852, 853, 855, 856, 863, 864, 6500, 872, 4970, 4971, 4973, 878, 4974, 881, 886, 888, 3961, 3450, 891, 893, 5503, 4992, 3970, 3971, 4995, 5508, 3974, 5509, 3977, 3978, 3979, 3980, 3981, 3982, 3984, 1940, 1941, 1942, 1943, 1944, 3993, 1946, 3994, 1948, 1949, 3995, 3996, 3999, 4000, 4001, 4002, 4003, 4004, 4005, 4006, 4007, 4008, 4010, 939, 4011, 4012, 4013, 4014, 4015, 4016, 4017, 5550, 949, 951, 4539, 7108, 7110, 974, 975, 5584, 5075, 980, 3032, 3033, 4589, 4590, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020, 1022, 1023} n=192
2024-03-26 12:04:05 [info     ] staging_sync.create_chart_revision chart_id=1024 slug=pig-yields-meat-per-animal-in-hectograms-100-grams
2024-03-26 12:04:05 [info     ] staging_sync.create_chart_revision chart_id=1025 slug=cattle-meat-yields-hectograms-100-grams-per-animal
2024-03-26 12:04:07 [info     ] staging_sync.create_chart_revision chart_id=1538 slug=global-change-over-the-last-50-years
2024-03-26 12:04:08 [info     ] staging_sync.create_chart_revision chart_id=1027 slug=cereal-distribution-to-uses
2024-03-26 12:04:09 [info     ] staging_sync.create_chart_revision chart_id=1029 slug=seafood-and-fish-production-thousand-tonnes
2024-03-26 12:04:10 [info     ] staging_sync.create_chart_revision chart_id=1030 slug=per-capita-meat-consumption-by-type-kilograms-per-year
2024-03-26 12:04:10 [info     ] staging_sync.create_chart_revision chart_id=1031 slug=per-capita-milk-consumption
2024-03-26 12:04:11 [warning  ] staging_sync.skip              chart_id=7685 reason=TARGET CHART DOESN'T HAVE 'isPublished'. slug=total-area-of-the-country-urbanization-and-land-area-faostat
2024-03-26 12:04:11,586 - [bugsnag] WARNING - No API key configured, couldn't notify
Traceback (most recent call last):
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/Users/prosado/.pyenv/versions/3.11.5/lib/python3.11/http/client.py", line 1286, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/prosado/.pyenv/versions/3.11.5/lib/python3.11/http/client.py", line 1332, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Users/prosado/.pyenv/versions/3.11.5/lib/python3.11/http/client.py", line 1281, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Users/prosado/.pyenv/versions/3.11.5/lib/python3.11/http/client.py", line 1041, in _send_output
    self.send(msg)
  File "/Users/prosado/.pyenv/versions/3.11.5/lib/python3.11/http/client.py", line 979, in send
    self.connect()
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
           ^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x15d595150>: Failed to establish a new connection: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 799, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=80): Max retries exceeded with url: /admin/api/charts (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x15d595150>: Failed to establish a new connection: [Errno 61] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/prosado/Documents/owid/repos/etl/.venv/bin/etl", line 6, in <module>
    sys.exit(cli())
             ^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/rich_click/rich_command.py", line 126, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/apps/staging_sync/cli.py", line 276, in cli
    resp = target_api.create_chart(target_chart.config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/apps/staging_sync/admin_api.py", line 38, in create_chart
    resp = requests.post(
           ^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/prosado/Documents/owid/repos/etl/.venv/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=80): Max retries exceeded with url: /admin/api/charts (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x15d595150>: Failed to establish a new connection: [Errno 61] Connection refused'))
pabloarosado commented 7 months ago

After https://github.com/owid/etl/pull/2458 , I don't get the error anymore, so I'll close this issue.