invalid twsInfo error

[paulbanyny]

I have been using IBC successfully with Gateway version 981 cor a while, however it has just begun throwing an error message after login: " invalid twcInfo " and I am unable to figure out what is causing it.

I had not changed any settings to make this error occur. I did try updating to IBC 3.11.1 Logging into IB Gateway manually with the same user info still works.

If there is any info you would like me to provide, please let me know.

[rlktradewright]

All I can suggest is make sure you've got the latest version of 981, and attach (don't quote) the IBC log file - it might reveal something, though probably not if you haven't changed anything. I'm pretty sure this is nothing to do with IBC.

I fired up Gateway 981 for my paper-trading account a few minutes ago and it was fine, so I can't reproduce this.

[robabram]

This issue is happening to me as well. To verify, I created a clean new Linux user on my computer and downloaded the latest versions of IB Gateway (981.3e) and IBC (3.11.1) releases. If I run IB Gateway without IBC, IB gateway runs fine. If I use IBC, the "invalid twsinfo" message popup appears about 1/3rd of the way in the progress bar of the loading window after login. I played with multiple settings in IBC, but the message always appears. I have attached my IBC log.

ibc-3.11.1_GATEWAY-981_Saturday.txt .

System: Fedora 35

[rlktradewright]

Well, I really don't know what to say, because I just created a new Ubuntu 20.04 virtual machine (my previous one having refused to boot after applying some updates!), installed TWS 981 and IBC 3.11.1, configured and ran the Gateway and it all worked fine.

I can't see anything wrong in your logfile. Admittedly Ubuntu is not Fedora but I'd be really surprised if there was any difference that would affect this.

So I have no information to proceed with.

[noneper]

This issue is happening to me as well. My logs shows below: left, connected normally at Wednesday, right, can't connected at Friday:

I always use the IBC(Gateway) but it didn't work from last Friday.

But, I try to execute the IBC(TWS), it can normally connect to the the server. :)

So, maybe IB change the returned information in IBGateway same as TWS?

Thanks so much for you any help.

[rlktradewright]

I think this must be an IB problem. There is certainly nothing IBC can do about it.

I notice that today only the 'Latest' version of Gateway (10.11.2f) is available on IB's website: the 'Stable' version doesn't seem to be there. It may be that they're aware of this problem and have pulled the 'Stable' version until they've fixed it.

I installed the 'Latest' Gateway on Linux and it was fine. I suggest you do the same and see if it works better for you.

[ppetershagen]

I got the same twsinfo error message with a long running 10.10.2s build today. The ssl issue I got afterwards was connected to a newly Ubuntu base image, without updated ca-certificates.

[ppetershagen]

So, I get the same error with 3.11.1 and the latest 10.11.2f on my freshly re-built Ubuntu based docker image. Interestingly, it did not yet happen for the live account I'm using and also fine for another paper trading account with the same docker image. Which sounds to me like there is something problematic with the specific account, maybe an old IB setting or so. I will attach the (IB+IBC merged) logs for the problematic paper trading account:

twsinfo_error.log

After deploying and testing the newly created image (with 10.11.2f) I can confirm, the gateway login doesn't fail for those 2 other user (1 live and 1 demo, both different accounts). It only fails for the above mentioned paper user login, which I use for testing. If you need any more logs, please let me know.

[rlktradewright]

@ppetershagen Thanks for that. A cursory glance at it doesn't tell me anything new, and I've run out of time for today. I probably won't be able to do any more on this until Thursday because of other commitments.

In any case, it certainly doesn't seem to be caused by anything IBC does, and there's nothing IBC can do to fix it, so it might be worthwhile sending your IB logfiles to them.

[rlktradewright]

I've just published release 3.12.0-beta.1 of IBC, which I hope will resolve this problem: so if you're currently stuck, please try that release and let me know what happens. Make sure you read the release description which tells you what to do.

Now, I really won't be able to do anything more till Thursday, so let's hope the release helps!

[ppetershagen]

Just tried out the beta, it didn't fix it in my docker env, logs attached. As my build script re-creates the IB and IBC folders from scratch, I didn't adjust anything else. My understanding of the release comments didn't indicate I have to for the environment I compose.

twserror_3.12.0-beta1.log

[ppetershagen]

Just tested 3.12.0-beta.2, it fixes it for my environment. Thanks Richard :+1:

[rlktradewright]

Oh wow, that's great to hear. Let's hope it works for others too.

[robabram]

I can also confirm that the beta release resolved the issue on my Fedora instance. Thank you very much!

[mlvn23]

I'm on IBC 3.8.7 and IB Gateway 981.3c and I've started seeing this error in one of my accounts. I copy and pasted the changes from a98626cf12264a804f64dcf743de76f0eaf3feb9 and the problem went away. Thanks!

[asustado89]

@rlktradewright Thank you very much! Works with IBC 3.12.0-beta.2 and IB Gateway 981.3d

[heshriti]

I had the same error. Tried to download and install the new version. Strangely, I am unable to launch IBC. In fact, when run the command in vnc mode, it simply returns without doing anything, no dialog, no error message. Something like echo "". Do you know where/how to force start ibc nd popup the banner etc.

[EswarKakani]

Works with IBC 3.12.0-beta.2 and IB Gateway 981.3c

[ValeryNo]

Works with TWS Version 1011 + IBC 3.12.0-beta.2.

How stupid of me was to ignore this update.. I saw new releases but thought - well it didn't happen to me so I'm fine, and the god damn thing broke today so I had to do emergency upgrade.

Thank you so much so the work you do!

Val

[rlktradewright]

@heshriti

I can only assume you've made some mistake in your setup, because there is zero doubt that if you do it properly it will not just 'do nothing'.

So I suggest you start again from scratch, following the instructions in the User Guide. If you still can't make it work, then reply and attach (don't quote) your start script, your config.ini (obscure the user id and password), and perhaps directory listings of your ibc and ibc/scripts folders.

[skister]

@heshriti I had a similar issue when I tried the new IBC and it flashed an X-window very briefly when I ran gatewaystart.sh then exited. When I ran gatewaystart.sh -inline, I saw the error message: ./gatewaystart.sh: line 200: exec: .../scripts/displaybannerandlaunch.sh: cannot execute: Permission denied

Make sure you chmod a+x all of the .sh files

[heshriti]

@skister Thank you. I am able to launch gateway. The server problem too went away. I guess this was a Sat bounce issue.

[noneper]

@rlktradewright, thank you so much for so quick beta! Nice works! 👍

And another routine here:

I try to find the solution in last whole week and did a a lots of things on my Windows Server 2016 by those methods I found from internet. But all of them failed.

But just then, I got success! I have to use TWS instead of IB Gateway in last whole week. But TWS sometimes hanging on when it shutdown daily. This prevented TWS(IBC) starting on next day through the Task Scheduler.

Now I changed a thing on my Win Server: I found the version of the java's JDK that IB Gatway/TWS use is 1.8.0_152. Some information mentioned bugs solved by higher version. Then I found the highest version be free is 1.8.0_202.

I downloaded this version and install on my Win Server, then copy the directory jre1.8.0_202 to the same location of 1.8.0_152-tzdata2019c, then switch the names of these two directories.

I hope IB Gateway use new version of Java JRE to run. And...

Unbelievable, it works! Now, IBC(Gateway) can directly connect again.

I am using the last stable version IB Gateway 9.8.1e with IBC 3.11.1 on my Windows Server 2016.

May above could help someone else. : )

--Note: Because on Windows, IB Gateway use the software install4j to install java's JDK. I didn't find how to change the setting of java path IB Gateway use, so just instead the files of in same place.

[deanb2]

Is there any chance that IBC could be causing an issue with Xtightvnc server?

I know it is unlikely, but we updated IBC yesterday, and started seeing huge cpu and memory use for Xtightvnc.  - 2GB or more instead of a few KB. IBC log file is pretty quiet.

The application runs in docker containers running Ubuntu with Xtightvnc as the display and Gateway 981.3c and the new IBC beta. Unfortunately the application was also updated, so we are investigating that as well.

If anyone is running in a similar environment, could you check the Xtightvnc process to see if it is running unusually high cpu and memory?

Thanks,

Dean Bennett

[rlktradewright]

@deanb2 I'd say there's zero chance of IBC being the cause. IBC has no user interface, and does nothing whatsoever involving text or graphics, apart from writing text messages to its logfile.

Most of the time IBC is doing nothing. It only springs into action when TWS/Gateway does something with a window (for example display the login dialog); or when certain timer events occur (for example saving settings on a schedule; or when responding to a command received by its command server (for example the STOP command). It is purely event driven, and has no 'program loop'.

So it's hard to conceive of a way that it could influence an external process like Xtightvnc.

[rlktradewright]

@noneper You can override the default JRE when you start IBC by setting the JAVA_PATH variable in StartTWS.bat or StartGateway.bat.

I'd be interested to know which bugs solved by the higher version of Java were causing the problem when TWS was shutting down. Please give details.

[deanb2]

I didn't think so.  Although Xtightvnc isn't exactly an unrelated external process - it is the X11 display for the Gateway. This setup has been running for several years, and this problem has never happened before

Thanks, Dean

On 12/11/2021 11:57 AM, Richard L King wrote:

@deanb2 https://github.com/deanb2 I'd say there's zero chance of IBC being the cause. IBC has no user interface, and does nothing whatsoever involving text or graphics, apart from writing text messages to its logfile.

Most of the time IBC is doing nothing. It only springs into action when TWS/Gateway does something with a window (for example display the login dialog); or when certain timer events occur (for example saving settings on a schedule; or when responding to a command received by its command server (for example the STOP command). It is purely event driven, and has no 'program loop'.

So it's hard to conceive of a way that it could influence an external process like Xtightvnc.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/IbcAlpha/IBC/issues/151#issuecomment-991748774, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALAAGIW5ZTSP6WK2FKN3ELUQONITANCNFSM5JJZYD6A. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[brndnmtthws]

I just started getting this today, after updating to the current stable standalone. It seems like an issue on IBKR's end, but I wanted to mention it here anyway in case anyone else just ran into it.

Oddly, it only happens with the Linux build, and not the macos one. I would have assumed the Java bytecode is equivalent across their releases, but who knows. One thing I didn't try yet is running the macos jars under Linux, maybe I will do that just to see what happens.

EDIT: Just to follow up, the jars in the Linux and macos builds have completely different checksums, so they're not even the same code. I guess the folks at IBKR didn't get the memo about "write once, run anywhere".

[rlktradewright]

I just installed TWS 1012.2u on an Ubuntu 20.04 virtual machine and it ran with no problems under IBC 3.14.0.

Can you please attach your IBC logfile?

[brndnmtthws]

Still happening today (logs contain 2 runs):

ibc.log launcher.log

[rlktradewright]

That's no good to me.

When you run IBC through the 'official' scripts, it creates a logfile and displays its name prominently in a a terminal window. This logfile contains a massive amount of useful information that is helpful in diagnosing what's going on.

But presumably you're running IBC via ib_insync, and I've no idea what that does, or where it records the information that IBC logs (which it writes to stdout, and the environment determines where that goes). But one line that IBC logs looks like this:

2022-09-08 22:15:01:793 IBC: Version: 3.14.0

so perhaps you can find that line and attach the file that contains it (I assume that this info is recorded somewhere).

If not, at least just tell me the IBC version you're using. If it's not 3.14.0, I suggest you upgrade to it.

[brndnmtthws]

But presumably you're running IBC via ib_insync, and I've no idea what that does, or where it records the information that IBC logs (which it writes to stdout, and the environment determines where that goes). But one line that IBC logs looks like this:

That's correct! I'll spend some more time putzing around. This method still uses ibcstart.sh, which you can see here: https://github.com/erdewit/ib_insync/blob/19f44c4f29efc0fe6ff8694a570847c66e7d6a87/ib_insync/ibcontroller.py#L142-L144

EDIT: Also I can confirm I'm using 3.14. The relevant code is here:

https://github.com/brndnmtthws/thetagang/blob/main/extract-installer.sh – fetches the TWS installer https://github.com/brndnmtthws/thetagang/blob/main/Dockerfile – installs IBC 3.14 plus other dependencies https://github.com/brndnmtthws/thetagang/blob/main/entrypoint.bash – launches the codes

[brndnmtthws]

Here's the output of the IBC script

[rlktradewright]

Ok, thanks for that.

The Error initializing QuantumRenderer: no suitable pipeline found is troubling. I don't get that on my Ubuntu.

I've seen it before and I think it is something to do with JavaFX not being correctly installed. I presume ib_insync uses the JRE that is installed with TWS?

I'm afraid I can't do any more on this just now, as I have to go out for a few hours. But I certainly think this is an environmental issue rather than an IBC issue, as it works fine on mine.

[rlktradewright]

I've just noticed that your logfile contains this:

--java-path = /opt/java/openjdk/bin

That will be the source of the problem. TWS doesn't work with OpenJDK.

[brndnmtthws]

I've just noticed that your logfile contains this:

--java-path = /opt/java/openjdk/bin

That will be the source of the problem. TWS doesn't work with OpenJDK.

It does indeed work with OpenJDK, as I've been running it for well over a year with OpenJDK (because the ancient busted JRE they ship doesn't work properly either). Other versions work fine, just not the current "stable" Linux build provided by IBKR. They must have recently introduced some changes that broke it. Whether they were intentional or not I don't know, but given that they ship different jars depending on the platform, I don't think the folks at IBKR understand how java works.

I hope they're not doing something dumb in the code like checking that the JVM is their "approved" special snowflake JVM. It would not surprise me if they were doing that.

[rlktradewright]

Well I don't know what version of things you've been using for the past year, but current versions of TWS use JavaFX for the login dialogs, and OpenJDK doesn't have a complete JavaFX implementation.

I completely disagree with what you say about 'ancient busted JRE'. The JRE that's installed by the IB installers is the one that they use to develop the product, so of course it works properly with that. That's precisely why it's included in the install, because it avoids people having to waste time wondering why it doesn't work when they think they know best and use a different JRE.

So try switching to the JRE that IB want you to use. I'd be extremely surprised if you still get this error.

[brndnmtthws]

Well I don't know what version of things you've been using for the past year, but current versions of TWS use JavaFX for the login dialogs, and OpenJDK doesn't have a complete JavaFX implementation.

I completely disagree with what you say about 'ancient busted JRE'. The JRE that's installed by the IB installers is the one that they use to develop the product, so of course it works properly with that. That's precisely why it's included in the install, because it avoids people having to waste time wondering why it doesn't work when they think they know best and use a different JRE.

So try switching to the JRE that IB want you to use. I'd be extremely surprised if you still get this error.

JavaFX is part of the OpenJDK project, FYI: https://openjdk.org/projects/openjfx/

Java 8 included it as part of the dist, but in current releases it's a separate project. Java 8 is technically deprecated, but for whatever reason some people refuse to upgrade.

In any case, it does indeed work with the latest Java and the official JavaFX aka OpenJFX. However for some reason they likely introduced some code to make it not work (like if (java version is not the one I like) { explode myself! }).

Regardless, I will stick with older releases for now until I figure out a way to make it work. Maybe they have some employees trolling these threads and can see that it's annoying as hell and they should just make the jars available (and working with the current LTS Java) without all the installer cruft.

[brndnmtthws]

@rlktradewright Thanks for the help, by the way. I realize I am being difficult, but being able to run TWS on aarch64 saves me money every month, which is why I go to the difficulty of making it work. I could just pay a few more bucks per month for an x86-64 VM, but that feels like losing.

FWIW the aarch64 macos version works fine (with OpenJDK, too), so what's up with that? I could just copy the macos build over, but that is a pain because it doesn't really integrate with the github actions workflow.

[brndnmtthws]

To follow up on my previous comments (in case anyone else bumps into this): looks like I was wrong, the issue appears to have been introduced in one of the latest builds of OpenJDK. For the time being, downgrading works.

The issue seems to have been introduced in this change specifically: https://github.com/docker-library/official-images/commit/b305ab5b435d4650d101df95c5877972e397911f, when upgrading from 17.0.4 to the 17.0.4.1 point release.

Pinning to 17.0.4 (as I did here) resolves it and works fine on arm64 (without the bundled JVM). It's pretty hard to tell what broke from the release notes.