search

Voyz / ibeam

IBeam is an authentication and maintenance tool used for the Interactive Brokers Client Portal Web API Gateway.
Apache License 2.0
522 stars 105 forks source link

IBeam Issue with IBKR Secondary Login (For Primary User) #190

Open Adam-Lechnos opened 1 month ago

Adam-Lechnos commented 1 month ago

Describe the bug When using IBeam with a secondary user (for a primary user within IBKR User Right & Access settings), IBeam does not seem to work. Are specific permissions required for API access to work when creating the secondary IBKR login? The issue above does not occur when using the non-headless clientportal.gw program directly. Here is the error I am getting within the docker container:

To Reproduce Steps to reproduce the behavior:

  1. Create a Secondary Login for Primary User within IBKR Settings --> User Access & Rights menu
  2. Update the env.list docker environment file with the new username and password
  3. Start the container via docker compose up -d command
  4. Tail the Ibeam container logs docker logs --follow ibeam
  5. Look for errors aligning with the screenshots below (login succeeds but with NO SESSION Status then fails post ERR CONNECTION REFUSED) b5vbnckzwu3d1 Screenshot 2024-06-02 192950

Expected behavior docker logs will show a successful connection and an AUTHENTICATED Status session per the example screenshot via my main login

Screenshot 2024-06-02 194001

Environment IBeam version: 0.5.1 Docker image or standalone: Docker Python version (standalone users only): OS: Ubuntu Linux

Additional context Main IBKR login works as expected. This only occurs when logging in with a secondary login. The issue above does not occur when using the non-headless clientportal.gw program directly. Attempted to run on a virtual cloud host to no avail. Same output as above.

Suggest a Fix

Voyz commented 1 month ago

Hey @Adam-Lechnos thanks for the thorough description of your issue 👍 Let me address it:

When using IBeam with a secondary user (for a primary user within IBKR User Right & Access settings),

Just to clarify this at the start - I haven't used the Gateway using this secondary user you describe, hence I'll be doing some guesswork here.

Are specific permissions required for API access to work when creating the secondary IBKR login?

I'd suggest you address this with the IBKR support to get more clarity on this question.

login succeeds but with NO SESSION Status

This isn't an error. It is IBeam attempting to figure out if the session is authenticated. In most cases when IBeam starts and there is no running Gateway, it will observe that there is no session and report that line. However, there are cases when IBeam starts and there already is a running Gateway with an active session (and sometimes even an authenticated one). Hence, the first thing that we do is to check for the status, and report accordingly.

Should we discover that there isn't an authenticated session, we proceed to log in - which is exactly what happens and it is an expected behaviour.

ERR CONNECTION REFUSED

This is strange, however I'd observe that this is an error reported by Selenium (the framework we use to interact with the browser), not by IBeam nor IBKR servers. Hence, I don't think the root cause lays in using the secondary login, but in something else.

Could you please copy-paste a full log text from that IBeam session, rather than posting screenshots? I'd need to see the entirety of the log to be able to deduce at which point does this error occur.

Also, can you run it with the following env vars?

IBEAM_LOG_LEVEL=DEBUG
IBEAM_ERROR_SCREENSHOTS=True

And share the screenshot saved at the time of the error - if one is present. You can find more info on how to do it here: https://github.com/Voyz/ibeam/wiki/IBeam-Configuration#debugging

Many thanks 👍

Adam-Lechnos commented 1 month ago

I had it working yesterday after enabling 'TWS' trading access. Now today, post maintenance on IBKR's end, I am seeing this post Handling IB-Key promo display... line:

This is the Client Portal Gateway
for any issues, please contact api@ibkr.com
and include a copy of your logs
****************************************************
https://www.interactivebrokers.com/api/doc.html
****************************************************
Open https://localhost:5000 to login
App demo is available after you login under: https://localhost:5000/demo#/
2024-06-05 23:47:56,872|I| Gateway auth webpage loaded
2024-06-05 23:47:56,877|I| Login attempt number 1
2024-06-05 23:48:04,196|I| Submitting the form
2024-06-05 23:48:09,103|I| Handling IB-Key promo display...
2024-06-05 23:48:10,426|E| Error encountered during authentication 
Exception:
  File "/srv/ibeam/ibeam_starter.py", line 168, in <module>
    success, shutdown, status = client.start_and_authenticate()
  File "/srv/ibeam/src/gateway_client.py", line 53, in start_and_authenticate
    success, shutdown, status = self.strategy_handler.try_authenticating(request_retries=request_retries)
  File "/srv/ibeam/src/handlers/strategy_handler.py", line 85, in try_authenticating
    return self._authentication_strategy_B(status, request_retries)
  File "/srv/ibeam/src/handlers/strategy_handler.py", line 140, in _authentication_strategy_B
    return self._log_in(status)
  File "/srv/ibeam/src/handlers/strategy_handler.py", line 151, in _log_in
    success, shutdown = self.login_handler.login()
  File "/srv/ibeam/src/handlers/login_handler.py", line 443, in login
    self.attempt(targets, wait_and_identify_trigger, driver)
  File "/srv/ibeam/src/handlers/login_handler.py", line 379, in attempt
    trigger, target = self.step_handle_ib_key_promo(targets, wait_and_identify_trigger, trigger)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/ibeam/src/handlers/login_handler.py", line 286, in step_handle_ib_key_promo
    ib_promo_key_trigger.click()
  File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webelement.py", line 93, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webelement.py", line 394, in _execute
    return self._parent.execute(command, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 344, in execute
    self.error_handler.check_response(response)
  File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
    raise exception_class(message, screen, stacktrace)

  <class 'selenium.common.exceptions.ElementClickInterceptedException'> Message: element click intercepted: Element is not clickable at point (392, 846)
  (Session info: chrome-headless-shell=120.0.6099.224)
Stacktrace:
#0 0x55c96d144233 <unknown>
#1 0x55c96ce0e0fe <unknown>
#2 0x55c96ce5f249 <unknown>
#3 0x55c96ce5ceb6 <unknown>
#4 0x55c96ce5a640 <unknown>
#5 0x55c96ce59914 <unknown>
#6 0x55c96ce4c62a <unknown>
#7 0x55c96ce85422 <unknown>
#8 0x55c96ce4bede <unknown>
#9 0x55c96ce856ee <unknown>
#10 0x55c96cea2680 <unknown>
#11 0x55c96ce851a3 <unknown>
#12 0x55c96ce4a13e <unknown>
#13 0x55c96ce4b712 <unknown>
#14 0x55c96d119534 <unknown>
#15 0x55c96d11c759 <unknown>
#16 0x55c96d11c238 <unknown>
#17 0x55c96d11cc05 <unknown>
#18 0x55c96d10babf <unknown>
#19 0x55c96d11cf8e <unknown>
#20 0x55c96d0f5c86 <unknown>
#21 0x55c96d1353e5 <unknown>
#22 0x55c96d1355cb <unknown>
#23 0x55c96d1437bf <unknown>
#24 0x7f00dacffea7 start_thread

2024-06-05 23:48:10,432|I| Cleaning up the resources. Display: <pyvirtualdisplay.display.Display object at 0x7f325a3e6390> | Driver: <selenium.webdriver.chrome.webdriver.WebDriver (session="34a48fbdcf9818f1e9b03b1ae4dbbf4a")>
2024-06-05 23:48:11,021|I| Logging in failed
2024-06-05 23:48:11,723|I| Starting maintenance with interval 60 seconds

This could be do to ongoing system maintenance, will try again tomorrow.

Adam-Lechnos commented 1 month ago

Still encountering the issue above. Is this related to a previous issue https://github.com/Voyz/ibeam/issues/68 ?

Voyz commented 1 month ago

Hey @Adam-Lechnos - try the solution proposed here: https://github.com/Voyz/ibeam/issues/188

There's 0.5.4-rc2 version that should be able to handle this. Let me know

Adam-Lechnos commented 1 month ago

The issue seems to be random. TWS under Trading Permissions is required when granting the secondary user access rights. I recommend pairing this with IP Restrictions which may be applied via the primary user account to the new user.

I will take a look at the proposed solution per your last comment, within issue 188.

Log of it failing then working:

Open https://localhost:5000 to login
App demo is available after you login under: https://localhost:5000/demo#/
2024-06-07 20:34:21,888|I| Gateway auth webpage loaded
2024-06-07 20:34:21,896|I| Login attempt number 1
2024-06-07 20:34:31,105|I| Submitting the form
2024-06-07 20:34:37,400|I| Handling IB-Key promo display...
2024-06-07 20:34:38,909|E| Error encountered during authentication
Exception:
  File "/srv/ibeam/ibeam_starter.py", line 168, in <module>
    success, shutdown, status = client.start_and_authenticate()
  File "/srv/ibeam/src/gateway_client.py", line 53, in start_and_authenticate
    success, shutdown, status = self.strategy_handler.try_authenticating(request_retries=request_retries)
  File "/srv/ibeam/src/handlers/strategy_handler.py", line 85, in try_authenticating
    return self._authentication_strategy_B(status, request_retries)
  File "/srv/ibeam/src/handlers/strategy_handler.py", line 140, in _authentication_strategy_B
    return self._log_in(status)
  File "/srv/ibeam/src/handlers/strategy_handler.py", line 151, in _log_in
    success, shutdown = self.login_handler.login()
  File "/srv/ibeam/src/handlers/login_handler.py", line 443, in login
    self.attempt(targets, wait_and_identify_trigger, driver)
  File "/srv/ibeam/src/handlers/login_handler.py", line 379, in attempt
    trigger, target = self.step_handle_ib_key_promo(targets, wait_and_identify_trigger, trigger)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/ibeam/src/handlers/login_handler.py", line 286, in step_handle_ib_key_promo
    ib_promo_key_trigger.click()
  File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webelement.py", line 93, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webelement.py", line 394, in _execute
    return self._parent.execute(command, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 344, in execute
    self.error_handler.check_response(response)
  File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
    raise exception_class(message, screen, stacktrace)

  <class 'selenium.common.exceptions.ElementClickInterceptedException'> Message: element click intercepted: Element is not clickable at point (392, 779)
  (Session info: chrome-headless-shell=120.0.6099.224)
Stacktrace:
#0 0x55c276474233 <unknown>
#1 0x55c27613e0fe <unknown>
#2 0x55c27618f249 <unknown>
#3 0x55c27618ceb6 <unknown>
#4 0x55c27618a640 <unknown>
#5 0x55c276189914 <unknown>
#6 0x55c27617c62a <unknown>
#7 0x55c2761b5422 <unknown>
#8 0x55c27617bede <unknown>
#9 0x55c2761b56ee <unknown>
#10 0x55c2761d2680 <unknown>
#11 0x55c2761b51a3 <unknown>
#12 0x55c27617a13e <unknown>
#13 0x55c27617b712 <unknown>
#14 0x55c276449534 <unknown>
#15 0x55c27644c759 <unknown>
#16 0x55c27644c238 <unknown>
#17 0x55c27644cc05 <unknown>
#18 0x55c27643babf <unknown>
#19 0x55c27644cf8e <unknown>
#20 0x55c276425c86 <unknown>
#21 0x55c2764653e5 <unknown>
#22 0x55c2764655cb <unknown>
#23 0x55c2764737bf <unknown>
#24 0x7f65de8c6ea7 start_thread

2024-06-07 20:34:38,921|I| Cleaning up the resources. Display: <pyvirtualdisplay.display.Display object at 0x7f83d057a1d0> | Driver: <selenium.webdriver.chrome.webdriver.WebDriver (session="c33fbdf170244b4f5641569a7a324ff7")>
2024-06-07 20:34:39,938|I| Logging in failed
2024-06-07 20:34:40,353|I| Starting maintenance with interval 60 seconds
2024-06-07 20:35:40,312|I| Maintenance
2024-06-07 20:35:40,698|I| Gateway running and authenticated, session id: b20fa7db5e8eddfa87ad456b6563101a, server name: JifN14055
2024-06-07 20:36:40,304|I| Maintenance
2024-06-07 20:36:40,415|I| Gateway running and authenticated, session id: 1959fd2a72ebe6f3f516a5f362b3b9ca, server name: JifN14055
2024-06-07 20:37:40,305|I| Maintenance
2024-06-07 20:37:40,676|I| Gateway running and authenticated, session id: 239cde7a523192739dea512b50f50384, server name: JifN14055
2024-06-07 20:38:40,304|I| Maintenance
2024-06-07 20:38:40,407|I| Gateway running and authenticated, session id: 1b8d741d2407c1d3f8cffd5d897e5354, server name: JifN14055
Adam-Lechnos commented 1 month ago

Solution via issue 188 resolved the ib_key_promo issue.

Voyz commented 1 month ago

@Adam-Lechnos great, happy to hear! 👍