Closed BrianHHough closed 5 months ago
Yes, the IBC log is quite extensive, and it contains a lot of information that can be very helpful in diagnosing problems. It's far better to attach the entire logfile to your post rather than selectively trying to quote bits from it. It's much quicker for the poster to do this, and much easier for me, as I have full information at my disposal and it's much easier to search it, compare it with other logfiles, etc.
As it happens in this case your extract says exactly what's wrong: you need to set the DISPLAY variable. I presume you're trying to run IBC on a headless Linux machine? Since Gateway is a GUI application, you need to install something that can handle the X11 gui primitives: a commonly used solution is xvfb, often in conjunction with one of the vnc variants so that you can view the GUI remotely. There are articles online that will help you with this.
You might also have a read through this:
https://dimon.ca/how-to-setup-ibc-and-tws-on-headless-ubuntu-in-10-minutes/
By the way the message 'IBC: Can't find the Gateway entry point: ibgateway.GWClient.main. Gateway is not correctly installed.' is rather misleading here: it's not actually the case that the entry point wasn't found, just that when it was called an exception occurred. I'll change the message for future IBC releases.
By the way, that point of failure is the first time that IBC does anything with the Gateway itself: everything you see in the logfile before that is just getting ready for loading the Gateway, and it then fails as a result of the system having no GUI support.
Hi @rlktradewright — thanks for getting back to me, I really appreciate it! Sure — I've included the most recent full log file from when I try to run the script as well as some explanations of what I am trying to do now with recent updates I've made.
The blocker for me right now is this error included below: java.awt.AWTError: Can't connect to X11 window server using ':0' as the value of the DISPLAY variable.
which is confusing to me because I'm setting the variable, which is also referenced in those docs you shared above: https://dimon.ca/how-to-setup-ibc-and-tws-on-headless-ubuntu-in-10-minutes/. Do you know if I need to do something before I start the .ibcstart.sh
script that I'm not thinking of?
Some updates since last time:
x11vnc
as a GUI interfacing package for the belowDISPLAY
(see below) to get rid of the No X11 DISPLAY variable was set, but this program performed an operation which requires it
error which I got before in my previous message above.These are the steps I follow to get the log output below:
cd /opt/ibc/resources/scripts
export DISPLAY=:0
./ibcstart.sh 1019 --gateway --tws-path=/home/ec2-user/Jts --ibc-path=/opt/ibc --ibc-ini=/home/ec2-user/ibc/config.ini --user=MYUSERNAME --pw=MYPASSWORD --mode=paper --on2fatimeout=exit
Here is the full log output for reference:
================================================================================
Starting IBC version on 2023-04-26 at 04:48:45
Operating system: Linux ip-10-0-0-9.ec2.internal 6.1.19-30.43.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Mar 15 14:44:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Arguments:
TWS version = 1019
Program = Gateway
Entry point = ibcalpha.ibc.IbcGateway
--tws-path = /home/ec2-user/Jts
--tws-settings-path =
--ibc-path = /opt/ibc
--ibc-ini = /home/ec2-user/ibc/config.ini
--mode = paper
--java-path =
--user = ***
--pw = ***
--fix-user =
--fix-pw =
=================================
Generating the classpath
Classpath=/home/ec2-user/Jts/ibgateway/1019/jars/jackson-core-2.12.3.jar:/home/ec2-user/Jts/ibgateway/1019/jars/jts4launch-1019.jar:/home/ec2-user/Jts/ibgateway/1019/jars/jxbrowser-7.26.jar:/home/ec2-user/Jts/ibgateway/1019/jars/jxbrowser-linux64-7.26.jar:/home/ec2-user/Jts/ibgateway/1019/jars/jxbrowser-swing-7.26.jar:/home/ec2-user/Jts/ibgateway/1019/jars/locales.jar:/home/ec2-user/Jts/ibgateway/1019/jars/log4j-api-2.17.1.jar:/home/ec2-user/Jts/ibgateway/1019/jars/log4j-core-2.17.1.jar:/home/ec2-user/Jts/ibgateway/1019/jars/total-2020.jar:/home/ec2-user/Jts/ibgateway/1019/jars/twslaunch-1019.jar:/home/ec2-user/Jts/ibgateway/1019/jars/twslaunch-install4j-1.12.jar:/home/ec2-user/Jts/ibgateway/1019/.install4j/i4jruntime.jar:/opt/ibc/IBC.jar
Generating the JAVA VM options
Java VM Options=-Xmx768m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=20 -XX:ConcGCThreads=5 -XX:InitiatingHeapOccupancyPercent=70 -Dtwslaunch.autoupdate.serviceImpl=com.ib.tws.twslaunch.install4j.Install4jAutoUpdateService -Dchannel=latest -Dexe4j.isInstall4j=true -Dinstall4jType=standalone -DjtsConfigDir=/home/ec2-user/Jts
Determining the location of java executable
Location of java executable=/home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/bin
Renaming IB's TWS or Gateway start script to prevent restart without IBC
Starting Gateway with this command:
"/home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/bin/java" -cp "/home/ec2-user/Jts/ibgateway/1019/jars/jackson-core-2.12.3.jar:/home/ec2-user/Jts/ibgateway/1019/jars/jts4launch-1019.jar:/home/ec2-user/Jts/ibgateway/1019/jars/jxbrowser-7.26.jar:/home/ec2-user/Jts/ibgateway/1019/jars/jxbrowser-linux64-7.26.jar:/home/ec2-user/Jts/ibgateway/1019/jars/jxbrowser-swing-7.26.jar:/home/ec2-user/Jts/ibgateway/1019/jars/locales.jar:/home/ec2-user/Jts/ibgateway/1019/jars/log4j-api-2.17.1.jar:/home/ec2-user/Jts/ibgateway/1019/jars/log4j-core-2.17.1.jar:/home/ec2-user/Jts/ibgateway/1019/jars/total-2020.jar:/home/ec2-user/Jts/ibgateway/1019/jars/twslaunch-1019.jar:/home/ec2-user/Jts/ibgateway/1019/jars/twslaunch-install4j-1.12.jar:/home/ec2-user/Jts/ibgateway/1019/.install4j/i4jruntime.jar:/opt/ibc/IBC.jar" -Xmx768m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=20 -XX:ConcGCThreads=5 -XX:InitiatingHeapOccupancyPercent=70 -Dtwslaunch.autoupdate.serviceImpl=com.ib.tws.twslaunch.install4j.Install4jAutoUpdateService -Dchannel=latest -Dexe4j.isInstall4j=true -Dinstall4jType=standalone -DjtsConfigDir=/home/ec2-user/Jts ibcalpha.ibc.IbcGateway "/home/ec2-user/ibc/config.ini" *** *** paper
IBC Settings:
AcceptIncomingConnectionAction=accept
AcceptNonBrokerageAccountWarning=yes
AllowBlindTrading=yes
AllowTwsApiOrders=yes
ClosedownAt=23:00
CommandServerPort=1234
DismissNSEComplianceNotice=yes
DismissPasswordExpiryWarning=yes
ExistingSessionDetected=manual
ExistingSessionDetectedAction=primary
FIX=yes
FixLoginId=***
FixPassword=***
Friday=00:00-23:59
IbAutoClosedown=no
IbDir=/home/ec2-user/Jts
IbLoginId=***
IbPassword=***
Locale=en
LogComponents=activate
LogLevel=DEBUG
LogToConsole=yes
LogToFile=yes
MinimizeMainWindow=no
Monday=00:00-23:59
ReadOnlyApi=no
ReadOnlyLogin=no
Saturday=OFF
StoreSettingsOnServer=no
Sunday=OFF
SuppressInfoMessages=yes
Thursday=00:00-23:59
TradingMode=paper
Tuesday=00:00-23:59
UserConfirmationTimeout=90
Wednesday=00:00-23:59
[IBC]=
[Log]=
[TradingHours]=
[Tws]=
End IBC Settings
2023-04-26 04:48:45:371 IBC: Version: 3.16.1
System Properties
------------------------------------------------------------
jtsConfigDir = /home/ec2-user/Jts
java.runtime.name = Java(TM) SE Runtime Environment
sun.boot.library.path = /home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/lib/amd64
java.vm.version = 25.202-b08
java.vm.vendor = Oracle Corporation
java.vendor.url = http://java.oracle.com/
path.separator = :
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
file.encoding.pkg = sun.io
sun.java.launcher = SUN_STANDARD
sun.os.patch.level = unknown
java.vm.specification.name = Java Virtual Machine Specification
user.dir = /home/ec2-user/Jts
java.runtime.version = 1.8.0_202-b08
java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
java.endorsed.dirs = /home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/lib/endorsed
os.arch = amd64
java.io.tmpdir = /tmp
line.separator =
java.vm.specification.vendor = Oracle Corporation
os.name = Linux
sun.jnu.encoding = UTF-8
java.library.path = /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
java.specification.name = Java Platform API Specification
java.class.version = 52.0
sun.management.compiler = HotSpot 64-Bit Tiered Compilers
exe4j.isInstall4j = true
os.version = 6.1.19-30.43.amzn2023.x86_64
user.home = /home/ec2-user
user.timezone = Etc/UCT
java.awt.printerjob = sun.print.PSPrinterJob
file.encoding = UTF-8
java.specification.version = 1.8
java.class.path = /home/ec2-user/Jts/ibgateway/1019/jars/jackson-core-2.12.3.jar:/home/ec2-user/Jts/ibgateway/1019/jars/jts4launch-1019.jar:/home/ec2-user/Jts/ibgateway/1019/jars/jxbrowser-7.26.jar:/home/ec2-user/Jts/ibgateway/1019/jars/jxbrowser-linux64-7.26.jar:/home/ec2-user/Jts/ibgateway/1019/jars/jxbrowser-swing-7.26.jar:/home/ec2-user/Jts/ibgateway/1019/jars/locales.jar:/home/ec2-user/Jts/ibgateway/1019/jars/log4j-api-2.17.1.jar:/home/ec2-user/Jts/ibgateway/1019/jars/log4j-core-2.17.1.jar:/home/ec2-user/Jts/ibgateway/1019/jars/total-2020.jar:/home/ec2-user/Jts/ibgateway/1019/jars/twslaunch-1019.jar:/home/ec2-user/Jts/ibgateway/1019/jars/twslaunch-install4j-1.12.jar:/home/ec2-user/Jts/ibgateway/1019/.install4j/i4jruntime.jar:/opt/ibc/IBC.jar
user.name = ec2-user
channel = latest
java.vm.specification.version = 1.8
sun.java.command = ibcalpha.ibc.IbcGateway /home/ec2-user/ibc/config.ini *** *** paper
java.home = /home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64
sun.arch.data.model = 64
user.language = en
java.specification.vendor = Oracle Corporation
awt.toolkit = sun.awt.X11.XToolkit
java.vm.info = mixed mode
java.version = 1.8.0_202
java.ext.dirs = /home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/lib/ext:/usr/java/packages/lib/ext
sun.boot.class.path = /home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/lib/resources.jar:/home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/lib/rt.jar:/home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/lib/sunrsasign.jar:/home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/lib/jsse.jar:/home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/lib/jce.jar:/home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/lib/charsets.jar:/home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/lib/jfr.jar:/home/ec2-user/.local/share/i4j_jres/Oda-jK0QgTEmVssfllLP/1.8.0_202_64/classes
java.vendor = Oracle Corporation
file.separator = /
twslaunch.autoupdate.serviceImpl = com.ib.tws.twslaunch.install4j.Install4jAutoUpdateService
java.vendor.url.bug = http://bugreport.sun.com/bugreport/
install4jType = standalone
sun.io.unicode.encoding = UnicodeLittle
sun.cpu.endian = little
sun.cpu.isalist =
------------------------------------------------------------
2023-04-26 04:48:45:395 IBC: Using default settings provider: ini file is /home/ec2-user/ibc/config.ini
2023-04-26 04:48:45:395 IBC: Using default login manager: getting username and password from args
2023-04-26 04:48:45:396 IBC: Using default main window manager
2023-04-26 04:48:45:396 IBC: Using default trading mode manager: constructor parameter args: tradingMode=paper
2023-04-26 04:48:45:399 IBC: Using default config dialog manager
2023-04-26 04:48:45:414 IBC: CommandServer is starting with port 1234
2023-04-26 04:48:45:420 IBC: CommandServer listening on addresses: fe80:0:0:0:1085:afff:fe2f:6c8b%enX0,10.0.0.9,10.0.0.217,0:0:0:0:0:0:0:1%lo,127.0.0.1; port: 1234
2023-04-26 04:48:45:432 IBC: CommandServer started and is ready to accept commands
2023-04-26 04:48:45:443 IBC: Gateway
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2023-04-26 04:48:45:458 IBC: An exception has occurred:
java.awt.AWTError: Can't connect to X11 window server using ':0' as the value of the DISPLAY variable.
at sun.awt.X11GraphicsEnvironment.initDisplay(Native Method)
at sun.awt.X11GraphicsEnvironment.access$200(X11GraphicsEnvironment.java:65)
at sun.awt.X11GraphicsEnvironment$1.run(X11GraphicsEnvironment.java:115)
at java.security.AccessController.doPrivileged(Native Method)
at sun.awt.X11GraphicsEnvironment.<clinit>(X11GraphicsEnvironment.java:74)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at java.awt.GraphicsEnvironment.createGE(GraphicsEnvironment.java:103)
at java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:82)
at sun.awt.X11.XToolkit.<clinit>(XToolkit.java:132)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at java.awt.Toolkit$2.run(Toolkit.java:860)
at java.awt.Toolkit$2.run(Toolkit.java:855)
at java.security.AccessController.doPrivileged(Native Method)
at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:854)
at ibcalpha.ibc.IbcTws.createToolkitListener(Unknown Source)
at ibcalpha.ibc.IbcTws.load(Unknown Source)
at ibcalpha.ibc.IbcGateway.main(Unknown Source)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2023-04-26 04:48:45:467 IBC: Exiting with exit code=1100
IBC returned exit status 76
Renaming TWS or Gateway .exe file to original name
Gateway finished
For some extra context, I did some testing inside of the ibc/resources
folder to test the startup .sh scripts for gateway and tws, but seem to be getting errors around the display with xterm for some reason.
I'm in this directory /opt/ibc/resources
The files in this directory are:
'IBC (Gateway).lnk' Restart.bat StartTWS.bat gatewaystart.sh.bak twsstart.sh.bak
'IBC (TWS).lnk' SendCommand.bat Stop.bat gatewaystartmacos.sh twsstartmacos.sh
IBC.jar 'Start TWS Live (daily).xml' config.ini scripts userguide.pdf
README.txt StartGateway.bat gatewaystart.sh twsstart.sh version
When I type echo $DISPLAY
, the output is :0
I am trying to run ./twsstart.sh
and ./gatewaystart.sh
from this directory for testing, but I'm getting this error: xterm: Xt error: Can't open display: :0
and the terminal window is trying to execute the script so it's in a pending state (as in, it doesn't exit/fail), so I'd have to Ctrl+C to exit out of it... so it's at least trying to run which I think is a good thing at least.
My twsstart.sh
vars in the file are as follows:
TWS_MAJOR_VRSN=1019
IBC_INI=~/ibc/config.ini
TRADING_MODE=paper
TWOFA_TIMEOUT_ACTION=exit
IBC_PATH=/opt/ibc/resources
TWS_PATH=~/Jts
TWS_SETTINGS_PATH=
LOG_PATH=~/ibc/logs
TWSUSERID=
TWSPASSWORD=
JAVA_PATH=
HIDE=
What I said regarding the log file is:
"It's far better to attach the entire logfile to your post rather than selectively trying to quote bits from it."
But now you've quoted the whole damn thing, which makes this post unwieldly and is precisely what I don't want. So, please, in future attach, do not quote.. If you don't know how to attach a file, cast your eyes to the last line of the edit box where you type your reply, where it says 'Attach files by dragging & dropping, selecting or pasting them.'
Simply setting the DISPLAY variable is not enough: it sounds like you haven't installed xvfb, see https://dimon.ca/how-to-setup-ibc-and-tws-on-headless-ubuntu-in-10-minutes/#h.zgi9innn3xl4
Can I suggest you also read this very helpful post from Jürgen on the IBC User Group:
https://groups.io/g/ibcalpha/message/2065
which points out that it may be better to use TigerVNC rather than xvfb andx11vnc, because it combines those two functions and works well in a JavaFX context.
Great, thanks so much @rlktradewright! I'll try TigerVNC instead and see how that goes. Also, noted about the logs and what you mean... sorry about that! 😅 I understand the formatting you're expecting now for logs as attachments and not inline. Appreciate your responses!
Ubuntu 20.04 while I installed Trade WorkStation (stable version 1027) by default
https://github.com/IbcAlpha/IBC/blob/07cfe26877375d0a290474cab6ee8c7396fa577a/userguide.md?plain=1#L208-L213
and also just copy and edit ~/github/ibc/config.ini
following two lines, that is IbLoginId
, IbPassword
:
https://github.com/IbcAlpha/IBC/blob/07cfe26877375d0a290474cab6ee8c7396fa577a/resources/config.ini#L81-L88
ls ~/Jts/
data kfdbpklheklmoigebbgnpjbnahamfallggpcokbk 'Trader Workstation.desktop' uninstall
jars launcher.log tws xmlopt.dat
jts.ini messageurls.properties tws.vmoptions
Confront error when ./ibcstart.sh 1027
(base) <username>@Ubuntu:/opt/ibc/scripts$ ./ibcstart.sh 1027
================================================================================
Starting IBC version on 2024-02-11 at 17:29:10
Operating system: Linux Ubuntu 5.15.0-94-generic #104~20.04.1-Ubuntu SMP Tue Jan 16 13:34:09 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Arguments:
TWS version = 1027
Program = TWS
Entry point = ibcalpha.ibc.IbcTws
--tws-path =
--tws-settings-path =
--ibc-path =
--ibc-ini =
--mode =
--java-path =
--user =
--pw =
=========================== An error has occurred =============================
Error: Offline TWS/Gateway version 1027 is not installed: can't find jars folder
Make sure you install the offline version of TWS/Gateway
IBC does not work with the auto-updating TWS/Gateway
@yezhengli-Mr9
You haven't installed the offline version!
On IB's download page, click the red ink at the bottom of the page that says 'Offline TWS latest' or 'Offline TWS':
Hi IBC Community,
I'm trying to run the
./ibcstart.sh
script using the --gateway flag, and I am able to execute the script so far by starting the script and receive the following steps:and then I see this in the logging where it will end with a warning that
IBC: Can't find the Gateway entry point: ibgateway.GWClient.main. Gateway is not correctly installed.
yet, it was able to do all of these other elements and communications between IBC and IBGateway from what I understand.This is my
config.ini
file:Is anyone able to help me understand what is going wrong in my connecting to the IBGateway when I run this script to start the IBC?
The log is quite extensive, so I know it is running something, but is the IBGateway supposed to be running before starting this script or is there something that isn't configured correctly?
I'd really appreciate any ideas or things to look into to fix this 😊