search

gnzsnz / ib-gateway-docker

Docker image with IB Gateway/TWS and IBC
https://github.com/users/gnzsnz/packages/container/package/ib-gateway
MIT License
274 stars 50 forks source link

Create build for linux/arm64 platform to support Apple Silicon (M1, M2) #81

Open benjaminsingleton opened 8 months ago

benjaminsingleton commented 8 months ago

Is your feature request related to a problem? Please describe

There isn't a build that is compatible with modern Apple computers, which run on the platform linux/arm64.

$ docker pull --platform linux/arm64 ghcr.io/gnzsnz/ib-gateway:stable
stable: Pulling from gnzsnz/ib-gateway
no matching manifest for linux/arm64 in the manifest list entries

On my MacBook M2, I get odd errors when trying to run the image if I force pull it as a linux/amd64. The emulator that tries to run amd64 does not seem to be able to run ib-gateway.

2024-03-02 13:28:21 2024-03-02 18:28:21:668 IBC: Starting Gateway
2024-03-02 13:28:22 assertion failed [rem_idx != -1]: Unable to find existing allocation for shared memory segment to unmap
2024-03-02 13:28:22 (VMAllocationTracker.cpp:745 remove_shared_mem)
2024-03-02 13:28:22  /home/ibgateway/ibc/scripts/ibcstart.sh: line 504:    30 Trace/breakpoint trap   "$java_path/java" -cp "$ibc_classpath" $java_vm_options$autorestart_option $entry_point "$ibc_ini" ${mode}
2024-03-02 13:28:22 IBC returned exit status 133
2024-03-02 13:28:22 autorestart file not found
2024-03-02 13:28:22 
2024-03-02 13:28:22 Gateway finished
image

Describe the solution you'd like

A build that supports linux/arm64.

Describe alternatives you've considered

Tried to run as linux/amd64 but it didn't work.

Docker Desktop 4.27.2 (137060). Docker version 25.0.3, build 4debf41

wesget182 commented 8 months ago

Hey @benjaminsingleton, if you clone the aarch64 branch and build your own image locally, it should resolve this problem. I've successfully built and started my ibgateway using this method. Although, I'm currently having trouble connecting to it from within my api. 

2024-03-03 11:00:03 2024-03-03 04:00:03:896 IBC: Login attempt: 1
2024-03-03 11:00:03 2024-03-03 04:00:03:909 IBC: Click button: Paper Log In
2024-03-03 11:00:06 2024-03-03 04:00:06:370 IBC: detected frame entitled: IBKR Gateway; event=Lost focus
2024-03-03 11:00:06 2024-03-03 04:00:06:370 IBC: detected frame entitled: IBKR Gateway; event=Deactivated
2024-03-03 11:00:06 2024-03-03 04:00:06:370 IBC: detected frame entitled: Loading...; event=Activated
2024-03-03 11:00:06 2024-03-03 04:00:06:373 IBC: detected frame entitled: Loading...; event=Focused
2024-03-03 11:00:06 2024-03-03 04:00:06:373 IBC: detected frame entitled: Loading...; event=Opened
2024-03-03 11:00:06 2024-03-03 04:00:06:373 IBC: Found Gateway main window
2024-03-03 11:00:06 2024-03-03 04:00:06:374 IBC: Got main window from future
2024-03-03 11:00:06 2024-03-03 04:00:06:374 IBC: Invoking config dialog menu
2024-03-03 11:00:06 2024-03-03 04:00:06:954 IBC: detected frame entitled: Loading...; event=Lost focus
2024-03-03 11:00:06 2024-03-03 04:00:06:954 IBC: detected frame entitled: Loading...; event=Deactivated
2024-03-03 11:00:06 2024-03-03 04:00:06:954 IBC: detected frame entitled: Connecting to server (trying for another 19 seconds)...; event=Activated
2024-03-03 11:00:06 2024-03-03 04:00:06:954 IBC: detected frame entitled: Connecting to server (trying for another 19 seconds)...; event=Focused
2024-03-03 11:00:06 2024-03-03 04:00:06:954 IBC: detected frame entitled: Connecting to server (trying for another 19 seconds)...; event=Opened
2024-03-03 11:00:09 LogModuleConfigurator-Init: Log4j Ver2.x found on classpath
2024-03-03 11:00:09 LogModuleConfigurator-Init: LogModuleConfigurator initialized with Log4j Verd.x
2024-03-03 11:00:10 2024-03-03 04:00:10:168 IBC: detected frame entitled: Starting application...; event=Lost focus
2024-03-03 11:00:10 2024-03-03 04:00:10:176 IBC: detected dialog entitled: DU7748944 Trader Workstation Configuration (Simulated Trading); event=Opened
2024-03-03 11:00:10 2024-03-03 04:00:10:176 IBC: detected frame entitled: Starting application...; event=Deactivated
2024-03-03 11:00:10 2024-03-03 04:00:10:176 IBC: detected dialog entitled: DU7748944 Trader Workstation Configuration (Simulated Trading); event=Activated
2024-03-03 11:00:10 2024-03-03 04:00:10:176 IBC: detected dialog entitled: Trader Workstation Configuration (Simulated Trading); event=Focused
2024-03-03 11:00:10 2024-03-03 04:00:10:177 IBC: Got config dialog from future
2024-03-03 11:00:10 2024-03-03 04:00:10:177 IBC: detected dialog entitled: Trader Workstation Configuration (Simulated Trading); event=Lost focus
2024-03-03 11:00:10 2024-03-03 04:00:10:177 IBC: detected frame entitled: Starting application...; event=Closed
2024-03-03 11:00:10 2024-03-03 04:00:10:177 IBC: Login has completed
2024-03-03 11:00:10 2024-03-03 04:00:10:177 IBC: detected dialog entitled: Warning; event=Opened
2024-03-03 11:00:10 2024-03-03 04:00:10:177 IBC: Click button: I understand and accept
2024-03-03 11:00:10 2024-03-03 04:00:10:250 IBC: detected dialog entitled: Trader Workstation Configuration (Simulated Trading); event=Deactivated
2024-03-03 11:00:10 2024-03-03 04:00:10:250 IBC: detected dialog entitled: Warning; event=Activated
2024-03-03 11:00:10 2024-03-03 04:00:10:251 IBC: detected dialog entitled: Warning; event=Focused
2024-03-03 11:00:10 2024-03-03 04:00:10:255 IBC: Setting ReadOnlyApi
2024-03-03 11:00:10 2024-03-03 04:00:10:300 IBC: Read-Only API checkbox is already set to: false
2024-03-03 11:00:10 2024-03-03 04:00:10:318 IBC: Configuration tasks completed
2024-03-03 11:00:10 2024-03-03 04:00:10:318 IBC: Click button: OK
2024-03-03 11:00:10 2024-03-03 04:00:10:392 IBC: detected dialog entitled: Warning; event=Lost focus
2024-03-03 11:00:10 2024-03-03 04:00:10:392 IBC: detected dialog entitled: Warning; event=Deactivated
2024-03-03 11:00:10 2024-03-03 04:00:10:392 IBC: detected dialog entitled: Warning; event=Closed
2024-03-03 11:00:10 2024-03-03 04:00:10:393 IBC: detected dialog entitled: Trader Workstation Configuration (Simulated Trading); event=Closed
2024-03-03 11:00:11 2024-03-03 04:00:11:795 IBC: detected dialog entitled: Pending Tasks; event=Closed
gnzsnz commented 8 months ago

indeed aarch64 was an attempt to get this working on arm64/aarch64.

if you build localy ib-gateway using that branch, it should (kind of) work. there is no working GUI, but the API works. I have not fully tested the API, so i can't warranty anything.

the main problem is to get a working java environment for ibgateway/TWS. aarch64 branch is not too far way, but not there yet.

any help to get this working on arm64 is more than welcome.

gnzsnz commented 8 months ago

@wesget182 , aarch64 is using an outdated "latest" version. updating latest/Dockerfile IB_GATEWAY_VERSION to latest version might help.

if this fix your issue please let me know. i can update the branch. I have not included this branch on the "master" branch because i don't consider it stable.

wesget182 commented 8 months ago

Ahh actually I got everything working just fine. There were some internal issues with my code combined with my confusion regarding the remove Client *clientId* from the logs. Thanks!

benjaminsingleton commented 8 months ago

Thanks for the pointer @wesget182 @gnzsnz! I was able to successfully build the image, run ib-gateway, and execute trades on my paper trading account. I wouldn't consider this super thorough testing, but it seems like it works as you'd expect. It would be great if you'd consider incorporating this in the master branch, so that a linux/arm64 build is deployed during CI/CD. Thanks again.

gnzsnz commented 8 months ago

@benjaminsingleton i'm not planning to merge aarch64 into master. At least not in it's current status, I don't consider it's current status as stable.

I welcome any pull request to improve it's current status, I have made some progress but not enough to merge this into master.

I would like to have:

Amir-Inbar commented 2 weeks ago

@gnzsnz @benjaminsingleton

Solution Overview:

Port Mapping Adjustment: The original docker-compose setup specified port mappings with 127.0.0.1 (e.g., 127.0.0.1:4001:4003). This strict IP binding caused issues on Apple Silicon. By changing it to broader exposure (e.g., 4001:4003 without 127.0.0.1), connectivity was resolved, allowing the ib-gateway container to function properly.

Platform Specification in Docker Compose: Adding platform: linux/amd64 to the docker-compose.yml file ensured the container consistently ran in linux/amd64 mode, improving stability on Apple Silicon.

Updated Image Version: Switching to the latest image (ghcr.io/gnzsnz/ib-gateway:latest) also helped as it provided additional compatibility updates.

VNC Configuration: We configured a VNC viewer to connect to the IB Gateway, allowing interaction with its GUI through port 5900. All incoming requests to IB Gateway worked as expected.

Code Example: Here’s the modified docker-compose.yml section:

services: ib-gateway: image: ghcr.io/gnzsnz/ib-gateway:latest platform: linux/amd64 ports:

image

These changes allowed the container to run smoothly on Apple Silicon (M1/M2) Macs, enabling full functionality of IB Gateway.

Credit for a friend of my @ohadch

gnzsnz commented 2 weeks ago

@Amir-Inbar thanks for sharing this. i believe that you are not running the image in aarch64 architecture, but emulating amd64. which is fine but will not work on other aarch64 architectures like raspi or linux running on M1.

what branch #102 is trying to do is to build an image in aarch64 architecture.