Error After adding TSL Certificate

[w-jamil]

Unable to run after adding .jks and .pem files in input directory.

docker run -v input ... produces the following error 2024-08-22 09:12:29,284|I| TLS certificates found and will be used for verification 2024-08-22 09:12:29,285|I| Custom conf.yaml found and will be used by the Gateway Traceback (most recent call last): File "/srv/ibeam/ibeam_starter.py", line 65, in http_handler = HttpHandler( ^^^^^^^^^^^^ File "/srv/ibeam/src/handlers/http_handler.py", line 116, in init self.build_ssh_context() File "/srv/ibeam/src/handlers/http_handler.py", line 123, in build_ssh_context self.ssl_context.load_verify_locations(self.inputs_handler.cacert_pem_path) PermissionError: [Errno 13] Permission denied

IBeam version: latest Docker image or standalone: Docker Python version (standalone users only): OS: Linux

[Voyz]

Hey @w-jamil thanks for reporting your issue 👍

Can you share the contents of your conf.yaml? Make sure to redact out any sensitive data

[w-jamil]

ip2loc: "US"
proxyRemoteSsl: true
proxyRemoteHost: "https://api.ibkr.com"
listenPort: 5000
listenSsl: true
svcEnvironment: "v1"
sslCert: "cacert.jks"
sslPwd: "mypass"
authDelay: 3000
portalBaseURL: ""
serverOptions:
    blockedThreadCheckInterval: 1000000
    eventLoopPoolSize: 20
    workerPoolSize: 20
    maxWorkerExecuteTime: 100
    internalBlockingPoolSize: 20
cors:
    origin.allowed: "*"
    allowCredentials: false
webApps:
    - name: "demo"
      index: "index.html"
ips:
  allow:
    - 192.*
    - 131.216.*
    - 127.0.0.1
    - 0.0.0.0

[Voyz]

Right, you see this bit:

sslCert: "cacert.jks"
sslPwd: "mypass"

Is this corresponding to your cacets?

[w-jamil]

Yes, I generated the file and password using the instructions here

[Voyz]

Right, then the setup seems to be correct. How are you setting up the container? Permission denied makes me think that the container user does not have permissions to read your inputs directory possibly

[w-jamil]

inputs directory has conf.yaml, so that is read, but when I add two files, above error. Is there a specific recomended way to set up the container. I use:

docker run -v /root/inputs:/srv/inputs --env IBEAM_ACCOUNT=ABC --env IBEAM_PASSWORD=MY_PASSWORD -p 5000:5000 voyz/ibeam

[Voyz]

Gotcha, your command seems fine. And yes, if conf.yaml is read then it should have read permissions. I'm sorry, I don't know how else to help 😔 I'd suggest you to search for the permission denied error in relation to reading ssl certs from within a docker container

[w-jamil]

I mounted the certificate with the image and now I am getting:

2024-08-22 12:12:12,389|I| ############ Starting IBeam version 0.5.1 ############ 2024-08-22 12:12:12,390|I| TLS certificates found and will be used for verification 2024-08-22 12:12:12,390|I| Custom conf.yaml found and will be used by the Gateway 2024-08-22 12:12:12,393|I| Secrets source: env 2024-08-22 12:12:12,394|I| Health server started at port=5001 2024-08-22 12:12:12,394|I| Configuration: {'UNDEFINED': <object object at 0x7af3c1f142d0>, 'INPUTS_DIR': '/srv/inputs/', 'OUTPUTS_DIR': '/srv/outputs', 'GATEWAY_DIR': '/srv/clientportal.gw', 'CHROME_DRIVER_PATH': '/usr/bin/chromedriver', 'GATEWAY_STARTUP': 20, 'GATEWAY_PROCESS_MATCH': 'ibgroup.web.core.clientportal.gw.GatewayStart', 'MAINTENANCE_INTERVAL': 60, 'SPAWN_NEW_PROCESSES': False, 'LOG_LEVEL': 'INFO', 'LOG_TO_FILE': True, 'LOG_FORMAT': '%(asctime)s|%(levelname)-.1s| %(message)s', 'REQUEST_RETRIES': 2, 'REQUEST_TIMEOUT': 15, 'RESTART_FAILED_SESSIONS': True, 'RESTART_WAIT': 15, 'REAUTHENTICATE_WAIT': 15, 'HEALTH_SERVER_PORT': 5001, 'SECRETS_SOURCE': 'env', 'GCP_SECRETS_URL': None, 'GATEWAY_BASE_URL': 'https://localhost:5000', 'ROUTE_AUTH': '/sso/Login?forwardTo=22&RL=1&ip2loc=on', 'ROUTE_VALIDATE': '/v1/portal/sso/validate', 'ROUTE_REAUTHENTICATE': '/v1/portal/iserver/reauthenticate?force=true', 'ROUTE_AUTH_STATUS': '/v1/api/iserver/auth/status', 'ROUTE_TICKLE': '/v1/api/tickle', 'ROUTE_LOGOUT': '/v1/api/logout', 'USER_NAME_EL': None, 'PASSWORD_EL': 'NAME@@password', 'SUBMIT_EL': 'CSS_SELECTOR@@.btn.btn-lg.btn-primary', 'ERROR_EL': None, 'SUCCESS_EL_TEXT': 'TAG_NAME@@Client login succeeds', 'OAUTH_TIMEOUT': 15, 'PAGE_LOAD_TIMEOUT': 15, 'ERROR_SCREENSHOTS': False, 'MAX_FAILED_AUTH': 5, 'MIN_PRESUBMIT_BUFFER': 5, 'MAX_PRESUBMIT_BUFFER': 30, 'MAX_IMMEDIATE_ATTEMPTS': 10, 'IBKEY_PROMO_EL_CLASS': 'CLASS_NAME@@ibkey-promo-skip', 'AUTHENTICATION_STRATEGY': 'B', 'MAX_STATUS_CHECK_RETRIES': 15, 'MAX_REAUTHENTICATE_RETRIES': 3, 'UI_SCALING': 1.0, 'TWO_FA_EL_ID': 'ID@@twofactbase', 'TWO_FA_NOTIFICATION_EL': 'CLASS_NAME@@login-step-notification', 'TWO_FA_INPUT_EL_ID': 'ID@@chlginput', 'TWO_FA_HANDLER': None, 'STRICT_TWO_FA_CODE': True, 'TWO_FA_SELECT_EL_ID': 'ID@@sf_select', 'TWO_FA_SELECT_TARGET': 'IB Key', 'CUSTOM_TWO_FA_HANDLER': 'custom_two_fa_handler.CustomTwoFaHandler'} 2024-08-22 12:12:12,395|I| Gateway not found, starting new one... 2024-08-22 12:12:12,395|I| Note that the Gateway log below may display "Open https://localhost:[PORT] to login" - ignore this command. 2024-08-22 12:12:12,395|I| Starting Gateway as Linux process with params: ['bash', 'bin/run.sh', 'root/conf.yaml'] running runtime path : root:dist/ibgroup.web.core.iblink.router.clientportal.gw.jar:build/lib/runtime/* config file : root/conf.yaml 2024-08-22 12:12:12,403|I| Gateway started with pids: [13] 2024-08-22 12:12:12,405|I| Gateway running but not serving yet. Consider increasing IBEAM_GATEWAY_STARTUP timeout. Error: <urlopen error [Errno 111] Connection refused> 2024-08-22 12:12:12,406|I| Gateway connection established 2024-08-22 12:12:12,407|I| Gateway running but not serving yet. Consider increasing IBEAM_GATEWAY_STARTUP timeout. Error: <urlopen error [Errno 111] Connection refused> 2024-08-22 12:12:12,407|I| NO SESSION Status(running=True, session=False, connected=False, authenticated=False, competing=False, collision=False, session_id=None, server_name=None, server_version=None, expires=None) 2024-08-22 12:12:12,408|I| Authentication strategy: "B" 2024-08-22 12:12:12,408|I| No active sessions, logging in... 2024-08-22 12:12:12,408|I| Loading auth webpage at https://localhost:5000/sso/Login?forwardTo=22&RL=1&ip2loc=on WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by io.netty.util.internal.ReflectionUtil (file:/srv/clientportal.gw/build/lib/runtime/netty-common-4.1.15.Final.jar) to constructor java.nio.DirectByteBuffer(long,int) WARNING: Please consider reporting this to the maintainers of io.netty.util.internal.ReflectionUtil WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release -> mount demo on /demo Java Version: 11.0.20

---

version: a27ed42161ad96c53e715ca5c5e3e3fa4cff5262 Mon, 24 Apr 2023 15:41:53 -0400

---

This is the Client Portal Gateway for any issues, please contact api@ibkr.com and include a copy of your logs

---

https://www.interactivebrokers.com/api/doc.html

---

Open https://localhost:5000 to login App demo is available after you login under: https://localhost:5000/demo#/ 2024-08-22 12:12:21,482|I| Gateway auth webpage loaded 2024-08-22 12:12:21,482|I| Login attempt number 1 2024-08-22 12:12:26,818|I| Submitting the form 2024-08-22 12:12:27,668|I| Webpage displayed "Client login succeeds" 2024-08-22 12:12:28,668|I| Cleaning up the resources. Display: <pyvirtualdisplay.display.Display object at 0x7af3bfcbfb10> | Driver: <selenium.webdriver.chrome.webdriver.WebDriver (session="8d7baf5d73921faeecb9048aa0b166a4")> 2024-08-22 12:12:28,717|I| Logging in succeeded 2024-08-22 12:12:28,743|E| Unrecognised URLError or socket.timeout Traceback (most recent call last): File "/usr/local/lib/python3.11/urllib/request.py", line 1348, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/usr/local/lib/python3.11/http/client.py", line 1283, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/local/lib/python3.11/http/client.py", line 1329, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/local/lib/python3.11/http/client.py", line 1278, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/local/lib/python3.11/http/client.py", line 1038, in _send_output self.send(msg) File "/usr/local/lib/python3.11/http/client.py", line 976, in send self.connect() File "/usr/local/lib/python3.11/http/client.py", line 1455, in connect self.sock = self._context.wrap_socket(self.sock, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/ssl.py", line 517, in wrap_socket return self.sslsocket_class._create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/ssl.py", line 1075, in _create self.do_handshake() File "/usr/local/lib/python3.11/ssl.py", line 1346, in do_handshake self._sslobj.do_handshake() ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'localhost'. (_ssl.c:1002)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/srv/ibeam/src/handlers/http_handler.py", line 143, in _request response = self.url_request(url, method=method) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/srv/ibeam/src/handlers/http_handler.py", line 128, in url_request return urllib.request.urlopen(req, context=self.ssl_context, timeout=self.request_timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/urllib/request.py", line 216, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/urllib/request.py", line 519, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/urllib/request.py", line 536, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/urllib/request.py", line 496, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/local/lib/python3.11/urllib/request.py", line 1391, in https_open return self.do_open(http.client.HTTPSConnection, req, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/urllib/request.py", line 1351, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'localhost'. (_ssl.c:1002)>

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/srv/ibeam/src/handlers/http_handler.py", line 208, in _request raise RuntimeError('Unrecognised URLError or socket.timeout') from e RuntimeError: Unrecognised URLError or socket.timeout 2024-08-22 12:12:28,752|E| Repeatedly reauthenticating failed 3 times. Killing the Gateway and restarting the authentication process. 2024-08-22 12:12:28,752|I| NO SESSION Status(running=True, session=False, connected=False, authenticated=False, competing=False, collision=False, session_id=None, server_name=None, server_version=None, expires=None) 2024-08-22 12:12:29,757|I| Starting maintenance with interval 60 seconds 2024-08-22 12:13:29,758|I| Maintenance 2024-08-22 12:13:29,759|I| Gateway not found, starting new one... 2024-08-22 12:13:29,759|I| Note that the Gateway log below may display "Open https://localhost:[PORT] to login" - ignore this command. 2024-08-22 12:13:29,759|I| Starting Gateway as Linux process with params: ['bash', 'bin/run.sh', 'root/conf.yaml'] running

[Voyz]

The error says 'Hostname mismatch, certificate is not valid for 'localhost''. Can you try creating and providing a cacert that has 'localhost' added as DNS?