search

microbiomedata / metadata-template-transformation

ETL between NMDC submission portal study records and user facility templates
0 stars 0 forks source link

Implement use of bearer token authentication system for `nmdc-server` API #37

Closed sujaypatil96 closed 2 weeks ago

sujaypatil96 commented 2 weeks ago

Implementing use of the new bearer token authentication system which was setup to improve authentication practices around nmdc-server API.

mslarae13 commented 2 weeks ago

@sujaypatil96 , not sure if it's related. But I'm getting an error when I try to run..

metadata-template-transformation-py3.10smit684@WE49137 metadata-template-transformation % poetry run python etl.py --submission 81041da9-3b7a-40c9-a66b-5166ccb3367e --user-facility jgi_mg --mapper input-files/jgi_mg_header.json --output file-name_jgi.xlsx --unique-field samp_name 

Traceback (most recent call last):
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
    conn.connect()
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 653, in connect
    sock_and_verified = _ssl_wrap_socket_and_match_hostname(
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 806, in _ssl_wrap_socket_and_match_hostname
    ssl_sock = ssl_wrap_socket(
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 465, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 509, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Users/smit684/.pyenv/versions/3.10.4/lib/python3.10/ssl.py", line 512, in wrap_socket
    return self.sslsocket_class._create(
  File "/Users/smit684/.pyenv/versions/3.10.4/lib/python3.10/ssl.py", line 1070, in _create
    self.do_handshake()
  File "/Users/smit684/.pyenv/versions/3.10.4/lib/python3.10/ssl.py", line 1341, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='data-dev.microbiomedata.org', port=443): Max retries exceeded with url: /auth/refresh (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/etl.py", line 243, in <module>
    cli()
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/etl.py", line 232, in cli
    metadata_df = metadata_retriever.retrieve_metadata_records(unique_field)
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/etl.py", line 40, in retrieve_metadata_records
    refresh_response = requests.post(
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/Users/smit684/VSCodeProjects/metadata-template-transformation/.venv/lib/python3.10/site-packages/requests/adapters.py", line 698, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='data-dev.microbiomedata.org', port=443): Max retries exceeded with url: /auth/refresh (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)')))