search

jessecooper / pyetrade

Python E-Trade API Wrapper
GNU General Public License v3.0
205 stars 96 forks source link

SSL errors with Python 3.11 #85

Open ajgringo619 opened 1 year ago

ajgringo619 commented 1 year ago

Getting the following SSLV3_ALERT_HANDSHAKE_FAILURE errors trying to run the sample code. This is happening with both consumer and sandbox keys:

Traceback (most recent call last):
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1092, in _validate_conn
    conn.connect()
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/urllib3/connection.py", line 635, in connect
    sock_and_verified = _ssl_wrap_socket_and_match_hostname(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/urllib3/connection.py", line 774, in _ssl_wrap_socket_and_match_hostname
    ssl_sock = ssl_wrap_socket(
               ^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 459, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 503, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.pyenv/versions/3.11.3/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.pyenv/versions/3.11.3/lib/python3.11/ssl.py", line 1075, in _create
    self.do_handshake()
  File "/home/ajgringo619/.pyenv/versions/3.11.3/lib/python3.11/ssl.py", line 1346, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1002)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
urllib3.exceptions.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1002)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/urllib3/connectionpool.py", line 844, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.etrade.com', port=443): Max retries exceeded with url: /oauth/request_token (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1002)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajgringo619/Development/projects/Linux/investments/etrade-test.py", line 11, in <module>
    print(oauth.get_request_token())
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/pyetrade/authorization.py", line 60, in get_request_token
    self.session.fetch_request_token(self.req_token_url)
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/requests_oauthlib/oauth1_session.py", line 287, in fetch_request_token
    token = self._fetch_token(url, **request_kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/requests_oauthlib/oauth1_session.py", line 365, in _fetch_token
    r = self.post(url, **request_kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/requests/sessions.py", line 637, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ajgringo619/.local/share/virtualenvs/investments_3.11.3/lib/python3.11/site-packages/requests/adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.etrade.com', port=443): Max retries exceeded with url: /oauth/request_token (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1002)')))

After finding this thread - https://www.reddit.com/r/etrade/comments/1398aha/etrade_developer_api_fail/ - I am able to get it to work by downgrading to Python 3.9. Has anyone else experienced this?

Robert-Zacchigna commented 1 year ago

Currently running my scripts using 3.10.5 and i dont have this issue, however, i did find this stackoverflow post in relation to this SSLV3_ALERT_HANDSHAKE_FAILURE error.

Can you give this solution a try?

ajgringo619 commented 1 year ago

Thanks for the info. I'll give this a shot after the Morgan/Stanley merger is complete.

1rocketdude commented 1 year ago

my turn with the short stick...everything working until 2023-06-22. Coincidental with upgrade from python 3.11.3 to 3.11.4. HTTPSConnectionPool(host='api.etrade.com', port=443): Max retries exceeded with url: /oauth/request_token (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1002)'))) anyone else having trouble today? I saw that ajgringo619 was having trouble with 3.11.3, so I'm skeptical it could be python versioning.

1rocketdude commented 1 year ago

I confirmed that python 3.10 and 3.11 with their associated crypto libraries causes the problem. python3.9 and it's libraries do not experience the SSL problem. Next step is to document when libraries are involved in the oauth process for each version of python.

1rocketdude commented 1 year ago

python 3.9, 3.10, and 3.11 all use the same library versions in my code, so I'm at a loss why python3.9 works while 3.10 and 3.11 do NOT.

pyetrade | 1.4.1 oauthlib | 3.2.2 requests_oauthlib | 1.3.1 requests | 2.31.0 xmltodict | 0.13.0 jxmlease | 1.0.3 certifi | 2023.5.7 urllib3 | 2.0.3

Robert-Zacchigna commented 1 year ago

very interesting, again i dont have issues with python 3.10.5. My package version list is below (as outlined by @1rocketdude):

pyetrade | 1.4.1 oauthlib | 3.2.2 requests_oauthlib | 1.3.1 requests | 2.28.1 xmltodict | 0.13.0 jxmlease | 1.0.3 certifi | 2022.9.24 urllib3 | 1.26.12

Different version are highlighted in bold, it appears that i am still using some older packages for serving http requests.

I don't think its a python version issue, can you guys try downgrading your packages and test to see if it works? Its possible that one of these packages (or a combination of them) is causing the handshake error.

I guess trial and error to figure out which ones it might be.

Robert-Zacchigna commented 1 year ago

Nvm, that didnt take long, its the urllib3 package.

i first upgraded the urllib3 package to the latest version and my script failed with the handshake error instantly. I upgraded all the other packages (except urllib3) and my script worked as normal but updating it as well broke it again.

Looking at their github, it looks like they went through some SSL changes from 1.26.16 to 2.0.3.

Something has definitely changed in regards to how they are handling SSL that the pyetrade oauth requests are not doing, my gut is telling me its got something to do with OpenSSL. I'm not entirely sure what the root cause is, don't have time to dive deeper at the moment.

1rocketdude commented 1 year ago

confirmed. Thanks everyone for your investigative work. It is indeed the urllib3 package.

I downgrade to urllib3 1.26.16, which is the latest version of the 1.xx series. Now everything works with python 3.11.4.

I agree with Robert - no need to deep dive into why. Simply publicize that urllib3 2.x is the problem. Simple fix.

cheers, Tom

On Jun 29, 2023, at 10:46 AM, Robert @.***> wrote:

Nvm, that didnt take long its the urllib3 package.

i first upgraded the urllib3 package to the latest version and my script failed with the handshake error instantly. Looking at their github, it looks like they went through some SSL changes from 1.26.16 to 2.0.3.

Something has definitely changed in regards to how they are handling SSL that the pyetrade oauth requests are not doing. I'm not sure what it could be, don't have time to dive deeper at the moment.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

ajgringo619 commented 1 year ago

Currently running my scripts using 3.10.5 and i dont have this issue, however, i did find this stackoverflow post in relation to this SSLV3_ALERT_HANDSHAKE_FAILURE error.

Can you give this solution a try?

Sorry I haven't got back to you. I'm still stuck on waiting for my consumer keys to be activated, been a month now.

justnisar commented 9 months ago

Thank you. Can confirm that downgrading to urllib3 1.26.16 helped. I'm using Python 3.10

1rocketdude commented 9 months ago

yes - downgrade to the latest 1.26 version - I think 1.26.18 and it magically works again.

python version doesn’t matter.

On Nov 21, 2023, at 10:29 PM, Nisar @.***> wrote:

Thank you. Can confirm that downgrading to urllib3 1.26.16 helped. I'm using Python 3.10

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

Rohit9605 commented 3 months ago

ERROR: Could not find a version that satisfies the requirement urllib==1.26.16 (from versions: none) and ERROR: Could not find a version that satisfies the requirement urllib==1.26.18 (from versions: none) both don't work for me

Robert-Zacchigna commented 3 months ago

ERROR: Could not find a version that satisfies the requirement urllib==1.26.16 (from versions: none) and ERROR: Could not find a version that satisfies the requirement urllib==1.26.18 (from versions: none) both don't work for me

@Rohit9605 Its the urllib3 package (urllib3==1.26.18) not urllib

Robert-Zacchigna commented 3 months ago

So I took another look at this and it seems i was somehow able to get things to work intermittently but i would like others try and confirm if possible.

PYTHON_VERSION = 3.10

  1. I upgraded urllib3 to 2.2.1 (latest 2.x version)
    • CMD: pip install --upgrade urllib3
      • Reran my script and qot the same SSL error as above
  2. Then I upgraded requests to the latest version (2.32.2)
    • CMD: pip install --upgrade requests
    • Which then also upgraded the following packages to their respective latest versions:
      • idna==3.7
      • certifi==2024.2.2

After doing the above, I tried running my test script (simple get_quote call) a few times before i stopped getting SSL error (at least 5-10 times) and got the desired output.

I'm not sure if its something else related to my machine (i dont think I've changed anything else out of the ordinary) but the only other thing I've seen to check is your machines openssl version, however, i definitely haven't touched that in a long time (for both of my windows and linux machines).

Its not working 100%, i still get the SSL error sometimes (quite frequently tbh), but either way, please try the above and report back here so we can narrow it down. Thank you.

ognjen-it commented 3 weeks ago

I'm confirming that latest version of urllib3 (1.26.19) works fine with python 3.11.8. Upgrade to that version fixed my problem.