search

EGA-archive / ega-download-client

A Python-based EGA download client
Apache License 2.0
94 stars 52 forks source link

Time out error #98

Closed tropicalberto closed 3 years ago

tropicalberto commented 3 years ago

Hi,

I am testing the pyega3 client in my mac device, as well as on a ubuntu device, but I don't manage to run the test command pyega3 -t -d datasets.

My suspicion is that the port 8052 is not open, as I don't get an output when I run openssl s_client -connect ega.ebi.ac.uk:8052. Also when I try to open a python console and run r = requests.get("https://ega.ebi.ac.uk:8052/elixir/central/stats/load") I get a time out error.

You can find below the error message that I get run the test command of pyega3.

My question is, do you know if the EGA server is working properly? Or do I need to do extra steps in my account (opening ports?)

Thanks a lot in advance! 

`USER@MACHINE~/s/A/E/e/pyega3> pyega3 -t -d datasets 
[2020-12-07 14:10:22 +0100] 
[2020-12-07 14:10:22 +0100] pyEGA3 - EGA python client version 3.4.0 (https://github.com/EGA-archive/ega-download-client)
[2020-12-07 14:10:22 +0100] Parts of this software are derived from pyEGA (https://github.com/blachlylab/pyega) by James Blachly
[2020-12-07 14:10:22 +0100] Python version : 3.9.0
[2020-12-07 14:10:22 +0100] OS version : Darwin Darwin Kernel Version 19.6.0: Thu Oct 29 22:56:45 PDT 2020; root:xnu-6153.141.2.2~1/RELEASE_X86_64
[2020-12-07 14:10:22 +0100] Server URL: https://ega.ebi.ac.uk:8052/elixir/data
[2020-12-07 14:10:22 +0100] Session-Id: 3964844147
[2020-12-07 14:10:22 +0100] Starting new HTTPS connection (1): ega.ebi.ac.uk:8443
[2020-12-07 14:10:23 +0100] https://ega.ebi.ac.uk:8443 "POST /ega-openid-connect-server/token HTTP/1.1" 200 None
[2020-12-07 14:10:23 +0100] 
[2020-12-07 14:10:23 +0100] Authentication success for user 'ega-test-data@ebi.ac.uk'
[2020-12-07 14:10:23 +0100] Starting new HTTPS connection (1): ega.ebi.ac.uk:8052
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 60] Operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 353, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 181, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x1094ef310>: Failed to establish a new connection: [Errno 60] Operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 573, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='ega.ebi.ac.uk', port=8052): Max retries exceeded with url: /elixir/data/metadata/datasets (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x1094ef310>: Failed to establish a new connection: [Errno 60] Operation timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/pyega3", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/pyega3/pyega3.py", line 647, in main
    reply = api_list_authorized_datasets(token)
  File "/usr/local/lib/python3.9/site-packages/pyega3/pyega3.py", line 162, in api_list_authorized_datasets
    r = requests.get(url, headers=headers)
  File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='ega.ebi.ac.uk', port=8052): Max retries exceeded with url: /elixir/data/metadata/datasets (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x1094ef310>: Failed to establish a new connection: [Errno 60] Operation timed out'))
`
tropicalberto commented 3 years ago

Just a small update, now it seems that both ports 8443 and 8052 are working if I run openssl s_client -connect ega.ebi.ac.uk:8443; openssl s_client -connect ega.ebi.ac.uk:8052.

But still, if I run the test command pyega3 -d -t datasets, I get a server error:

[2020-12-08 08:44:28 +0100] 
[2020-12-08 08:44:28 +0100] pyEGA3 - EGA python client version 3.4.0 (https://github.com/EGA-archive/ega-download-client)
[2020-12-08 08:44:28 +0100] Parts of this software are derived from pyEGA (https://github.com/blachlylab/pyega) by James Blachly
[2020-12-08 08:44:28 +0100] Python version : 3.9.0
[2020-12-08 08:44:28 +0100] OS version : Darwin Darwin Kernel Version 19.6.0: Thu Oct 29 22:56:45 PDT 2020; root:xnu-6153.141.2.2~1/RELEASE_X86_64
[2020-12-08 08:44:28 +0100] Server URL: https://ega.ebi.ac.uk:8052/elixir/data
[2020-12-08 08:44:28 +0100] Session-Id: 1727143863
[2020-12-08 08:44:28 +0100] Starting new HTTPS connection (1): ega.ebi.ac.uk:8443
[2020-12-08 08:44:29 +0100] https://ega.ebi.ac.uk:8443 "POST /ega-openid-connect-server/token HTTP/1.1" 200 None
[2020-12-08 08:44:29 +0100] 
[2020-12-08 08:44:29 +0100] Authentication success for user 'ega-test-data@ebi.ac.uk'
[2020-12-08 08:44:29 +0100] Starting new HTTPS connection (1): ega.ebi.ac.uk:8052
[2020-12-08 08:46:14 +0100] https://ega.ebi.ac.uk:8052 "GET /elixir/data/metadata/datasets HTTP/1.1" 500 None
Traceback (most recent call last):
  File "/usr/local/bin/pyega3", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/pyega3/pyega3.py", line 647, in main
    reply = api_list_authorized_datasets(token)
  File "/usr/local/lib/python3.9/site-packages/pyega3/pyega3.py", line 163, in api_list_authorized_datasets
    r.raise_for_status()
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error:  for url: https://ega.ebi.ac.uk:8052/elixir/data/metadata/datasets

This is the output that I obtain when I run openssl s_client -connect ega.ebi.ac.uk:8052 : 

CONNECTED(00000005)
depth=2 C = BM, O = QuoVadis Limited, CN = QuoVadis Root CA 2 G3
verify return:1
depth=1 C = NL, O = QuoVadis Trustlink B.V., CN = QuoVadis Europe EV SSL CA G1
verify return:1
depth=0 jurisdictionCountryName = GB, businessCategory = Government Entity, serialNumber = Government Entity, C = GB, ST = Essex, L = Saffron Walden, O = European Bioinformatics Institute, CN = ega.ebi.ac.uk
verify return:1
---
Certificate chain
 0 s:/jurisdictionCountryName=GB/businessCategory=Government Entity/serialNumber=Government Entity/C=GB/ST=Essex/L=Saffron Walden/O=European Bioinformatics Institute/CN=ega.ebi.ac.uk
   i:/C=NL/O=QuoVadis Trustlink B.V./CN=QuoVadis Europe EV SSL CA G1
 1 s:/C=NL/O=QuoVadis Trustlink B.V./CN=QuoVadis Europe EV SSL CA G1
   i:/C=BM/O=QuoVadis Limited/CN=QuoVadis Root CA 2 G3
 2 s:/C=BM/O=QuoVadis Limited/CN=QuoVadis Root CA 2 G3
   i:/C=BM/O=QuoVadis Limited/CN=QuoVadis Root CA 2 G3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIH+DCCBeCgAwIBAgIUUNfgpHyIRZ1XErfmVXRlZ+ZvvuQwDQYJKoZIhvcNAQEL
BQAwVjELMAkGA1UEBhMCTkwxIDAeBgNVBAoMF1F1b1ZhZGlzIFRydXN0bGluayBC
LlYuMSUwIwYDVQQDDBxRdW9WYWRpcyBFdXJvcGUgRVYgU1NMIENBIEcxMB4XDTIw
MDgyNTE1MjM1M1oXDTIyMDgyNTE1MzMwMFowgccxEzARBgsrBgEEAYI3PAIBAxMC
R0IxGjAYBgNVBA8MEUdvdmVybm1lbnQgRW50aXR5MRowGAYDVQQFExFHb3Zlcm5t
ZW50IEVudGl0eTELMAkGA1UEBhMCR0IxDjAMBgNVBAgMBUVzc2V4MRcwFQYDVQQH
DA5TYWZmcm9uIFdhbGRlbjEqMCgGA1UECgwhRXVyb3BlYW4gQmlvaW5mb3JtYXRp
Y3MgSW5zdGl0dXRlMRYwFAYDVQQDDA1lZ2EuZWJpLmFjLnVrMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoQU0WiIRn4yqVpZtKX3gXE8qmrq8exhmP+Js
I3hfSoLeQl7jfDi0t8d9w5hHY0zch4fUA8ndIOzGPSOmyEd/8V2YGWuxViYEMU8c
dG/DSQvKkIKt103H8Aw4InEZRF0L6wcWzEx+gS1eqaQ5PvqIDZ3Rsbm4yJSai6RG
geNc+Arw+iEH70kosPocedTL1f/2VYadUTOjNg9wSTrd3xzMCwMTblV/nDQRQzYH
kiE4k9h0nPo/Z3rlas4pDLtxy1GD7YXKcA5cKFRUmti7lH3zkEBODy0IsdPaWWl1
v6fkVnNoCPYwop6HjgXp+X06OruobSeo+kfu8lIR0g8ZCH0TbQIDAQABo4IDSjCC
A0YwDAYDVR0TAQH/BAIwADAfBgNVHSMEGDAWgBRJPQ7cecXL2xKj1Q2r8s+pr+Nm
qTCBgwYIKwYBBQUHAQEEdzB1MEcGCCsGAQUFBzAChjtodHRwOi8vdHJ1c3QucXVv
dmFkaXNnbG9iYWwuY29tL3F1b3ZhZGlzZXVyb3BlZXZzc2xjYWcxLmNydDAqBggr
BgEFBQcwAYYeaHR0cDovL29jc3AucXVvdmFkaXNnbG9iYWwuY29tMBgGA1UdEQQR
MA+CDWVnYS5lYmkuYWMudWswWgYDVR0gBFMwUTBGBgwrBgEEAb5YAAJkAQIwNjA0
BggrBgEFBQcCARYoaHR0cDovL3d3dy5xdW92YWRpc2dsb2JhbC5jb20vcmVwb3Np
dG9yeTAHBgVngQwBATAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUHAwEwSgYD
VR0fBEMwQTA/oD2gO4Y5aHR0cDovL2NybC5xdW92YWRpc2dsb2JhbC5jb20vcXVv
dmFkaXNldXJvcGVldnNzbGNhZzEuY3JsMB0GA1UdDgQWBBQIjAP3ZAbXIYcCYKR2
QoMynfiANjAOBgNVHQ8BAf8EBAMCBaAwggF9BgorBgEEAdZ5AgQCBIIBbQSCAWkB
ZwB1ACl5vvCeOTkh8FZzn2Old+W+V32cYAr4+U1dJlwlXceEAAABdCZBGDoAAAQD
AEYwRAIgGWqKLlao31gTB2FyN9aiVst0iNj0UGuosodhZl20YPUCIDGH619fHjM2
pbBTc6iGfiaur3mhUvML/Ocm6Jd6FPtXAHYARqVV63X6kSAwtaKJafTzfREsQXS+
/Um4havy/HD+bUcAAAF0JkEX/AAABAMARzBFAiEA20ZXX0mYKtAshMUjEFBX48T9
6rvdRmlAHFC2oBzGE+oCIA1/XDXxBHKEFMc9XkQlmcXDGCH0hW53hUd07dwDR8Eu
AHYAQcjKsd8iRkoQxqE6CUKHXk4xixsD6+tLx2jwkGKWBvYAAAF0JkEYZAAABAMA
RzBFAiBYqJE2B6+ohsEuCD3C89doyEUSRYrvkobwjFqyc8EfxQIhAKbWyaEDSQkd
5tON0ED58QQspGBEDH/faxCFT//8b+TdMA0GCSqGSIb3DQEBCwUAA4ICAQBh0LI5
S06TMe2TdQic9IXrZaVRhi2AmhZ/f6LwxHlY16EfqkuErIjrc3ADutSvqVRw33eI
zAEt3mxfWt/AaoeBW1s1P+Zce9nKY/3ven5q/k47zRgANQ3j0dhCJuGdMyAk/64H
YponUW4zZh6siX3YgrpOdS0ipZk/f2ANQIy7sQbr36RePHQVIOqUP/IMKIzR/eqE
KEUkZ3/RFV3/9PGnKCQo6BaejTc6uDpmjOpu5jhIO6RXCC25VGPscBtUD6qXL5Z+
7cdkfbTnfIlrXx/jkq8wUomHtyduSfXgbU9zKZnVuQKgkTo9hqrswvOtTTE0+kSp
UMCt0P/HtdSPXxUPI4bfqBbeElWh1VhX0n7Ljc6osXbhczcH4sNJaOV8B6Nr6cdM
fF/E/TwAGAcuunGPkxnEKoRjFQKefuYc8d0SW8bz8hecdsMm8AmrBurQ9WCqIUHt
CiSDAupLLZZDlo50Udu4BNB29AbGjxonGxhHn9+/5ORjymCy31GtyDRpmu3mm7mE
TmfqJf2+r84SE40vvnkeZbtEGFzN1PR4wdZyx8iI2zEkgFRA6T+M6Ia/toN3d1T9
KE1M0ryq/S5qECnj9u3wlVzibSO9IWH6tQpIGOFxht/1QWxDeCOPIwAcO3GI0Mky
/mGeXPOdycZvudzA+TOcZk0/iUtlevvhStrBcg==
-----END CERTIFICATE-----
subject=/jurisdictionCountryName=GB/businessCategory=Government Entity/serialNumber=Government Entity/C=GB/ST=Essex/L=Saffron Walden/O=European Bioinformatics Institute/CN=ega.ebi.ac.uk
issuer=/C=NL/O=QuoVadis Trustlink B.V./CN=QuoVadis Europe EV SSL CA G1
---
No client certificate CA names sent
---
SSL handshake has read 5311 bytes and written 514 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : AES256-GCM-SHA384
    Session-ID: FA103C6BCED430CFE2D8F917351DD852E255126C6AFCCBC817547BDC06AD2867
    Session-ID-ctx: 
    Master-Key: 6FAE4F09EEE353DA3A620596D0D22898ED04CD761E0D75E2D73EE9775BB392AA792DC15B7F6693E5426ADC805FF1C5C0
    Start Time: 1607413672
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---
malloryfreeberg commented 3 years ago

Hi @tropicalberto. Thank you for reporting this issue. I can see that pyega3 itself is working, since it is able to authenticate the test account successfully. Therefore, since this appears to be a service-related issue, if you are still experiencing this behavior please email EGA Helpdesk at helpdesk@ega-archive.org with this issue and attach the output log produced by pyega3 (pyega3_output.log). The Helpdesk team can then provide further assistance. Thank you, and apologies for the inconvenience.