Unknown Username.Password/Duplicate Session?

[longunmin]

I am having an issue similar to what was described at the end of #235, in that every night at 00:24 (or thereabouts), I get an error message saying that TWS can't login because of Unrecognized Username and Password. In reviewing the aforementioned issue, you say that its related to secondary instances. The other person was running in docker, vs my instance is installed via the recommended procedure. I am on Ubunutu 22.04 with major TWS version 10.28 and the most recent version of IBC. In paper trading mode currently. The only portion of the installation that wasn't done as recommended was to encrypt ibc folder. The interesting thing is that I have auto-restart set to 6:45PM EST in TWS (which IBC handles perfectly), but it appears that TWS is trying to restart again for some reason around midnight EST. I have no doubt that it is a user error on my end, I'm just trying to figure out how I can stop the attempts at duplicate instances. I am more than happy to "nuke it from space" and reinstall both IBC and TWS, but I wanted to see if there was a clear mistake that I made and/or a better solution.

I have attached the logs from IBC for the 3 most recent nights. Thank you for your time and help. ibc-3.18.0_TWS-1028_Monday.txt ibc-3.18.0_TWS-1028_Tuesday.txt ibc-3.18.0_TWS-1028_Wednesday.txt . Thank you for your time and help.

[grantrosse]

hey- I started #235, I narrowed it down to an issue with the paper trading mode. With the same setup using live trading I didn't have the issue anymore- can you check that?

[longunmin]

hey- I started #235, I narrowed it down to an issue with the paper trading mode. With the same setup using live trading I didn't have the issue anymore- can you check that?

Thank you! I can give that a try. I will close down/change out paper mode this evening and set it to live and see if I get the same unknown un/pw issue

[longunmin]

This is probably a stupid question, but how do I turn off 2FA?

[rlktradewright]

You can't turn off 2FA for live accounts. IBKR policy.

[longunmin]

You can't turn off 2FA for live accounts. IBKR policy.

Thanks! Looks like I don't receive the same issue when in Live mode. So seems like it's just a paper mode problem. Figure this is an edge case, but do you have any advice, as not ready to deploy my strat in live mode. Maybe rollback a version of TWS offline? And just to be clear, I'm talking about the Unknown Username and Password issue, not the 2FA

[rlktradewright]

Having spent hours looking at all this again, it seems to me that your problem is arising during the daily reset period. What is supposed to happen here is that TWS clears out the accumulated dross from the day, and automatically re-logs in to the account server, and then carries on sending out any market data that was running before the reset. Typically this takes a few seconds.

A question: are you quite sure that you don't have anything set up to automatically try and 'work around' the daily reset? Do your API client applications do anything 'special' at this time?

For me, the IBC logfile shows entries like this for both live and paper:

2024-03-21 05:27:18:916 IBC: detected frame entitled: U95482 Interactive Brokers; event=Lost focus
2024-03-21 05:27:18:919 IBC: detected frame entitled: U95482 Interactive Brokers; event=Deactivated
2024-03-21 05:27:18:920 IBC: detected frame entitled: Connecting to server...; event=Activated
2024-03-21 05:27:18:924 IBC: detected frame entitled: Connecting to server...; event=Focused
2024-03-21 05:27:18:924 IBC: detected frame entitled: Connecting to server...; event=Opened
2024-03-21 05:27:33:178 IBC: detected frame entitled: Starting application...; event=Closed
2024-03-21 05:27:33:179 IBC: detected frame entitled: Starting application...; event=Lost focus
2024-03-21 05:27:33:179 IBC: detected frame entitled: Starting application...; event=Deactivated
2024-03-21 05:27:33:179 IBC: detected frame entitled: U95482 Interactive Brokers; event=Activated
2024-03-21 05:27:33:181 IBC: detected frame entitled: U95482 Interactive Brokers; event=Focused

In this, the main window loses focus and is deactivated, a 'Connecting to server...' dialog is shown and changes its title to 'Starting application,,,' (which is TWS automatically getting going again after the reset), and then the main window is diaplayed again.

But your log looks like this at that point:

2024-03-21 00:23:06:613 IBC: detected frame entitled: Connecting to server...; event=Opened
2024-03-21 00:23:06:613 IBC: detected frame entitled: Connecting to server...; event=Activated
2024-03-21 00:23:06:672 IBC: detected frame entitled: Connecting to server...; event=Focused
2024-03-21 00:24:45:632 IBC: detected dialog entitled: Re-login is required; event=Opened
2024-03-21 00:24:45:633 IBC: Re-login to session
2024-03-21 00:24:45:633 IBC: Click button: Re-login
2024-03-21 00:24:45:710 IBC: detected frame entitled: Attempt 7: Authenticating...; event=Lost focus
2024-03-21 00:24:45:710 IBC: detected dialog entitled: Re-login is required; event=Closed
2024-03-21 00:24:45:711 IBC: detected frame entitled: Attempt 7: Authenticating...; event=Focused
2024-03-21 00:24:45:883 IBC: detected frame entitled: ; event=Lost focus
2024-03-21 00:24:45:883 IBC: detected frame entitled: ; event=Deactivated
2024-03-21 00:24:45:898 IBC: detected dialog entitled: Unrecognized Username or Password; event=Opened
2024-03-21 00:24:45:907 IBC: detected dialog entitled: Unrecognized Username or Password; event=Activated
2024-03-21 00:24:45:908 IBC: detected dialog entitled: Unrecognized Username or Password; event=Focused

So my TWS seems to be automatically continuing the session, whereas yours is demanding re-login.

Why the difference? As far as I can see, the only differences between your setup and mine are these:

• you have the Daily Lineup displayed earlier in the session, but that was closed hours before, so I doubt that has any influence
• you have MinimizeMainWindow=yes in config.ini; again I do't see why that should have any effect

Can I suggest you turn off the Daily Lineup, and remove the minimize setting, just to see if it makes any difference.

Also when you see the 'Unrecognized Username or Password' dialog, if you click the button does it redisplay the login dialog.? If so, if you re-enter your username and password, will it then login again correctly? I'm not averse to trying to make IBC re-enter the username and password and click Login in these circumstances, but I don't understand why it apparently logs you off, but for practically every other IBC user it just sails through with no problem.

[longunmin]

Having spent hours looking at all this again, it seems to me that your problem is arising during the daily reset period. What is supposed to happen here is that TWS clears out the accumulated dross from the day, and automatically re-logs in to the account server, and then carries on sending out any market data that was running before the reset. Typically this takes a few seconds.

A question: are you quite sure that you don't have anything set up to automatically try and 'work around' the daily reset? Do your API client applications do anything 'special' at this time?

For me, the IBC logfile shows entries like this for both live and paper:

2024-03-21 05:27:18:916 IBC: detected frame entitled: U95482 Interactive Brokers; event=Lost focus
2024-03-21 05:27:18:919 IBC: detected frame entitled: U95482 Interactive Brokers; event=Deactivated
2024-03-21 05:27:18:920 IBC: detected frame entitled: Connecting to server...; event=Activated
2024-03-21 05:27:18:924 IBC: detected frame entitled: Connecting to server...; event=Focused
2024-03-21 05:27:18:924 IBC: detected frame entitled: Connecting to server...; event=Opened
2024-03-21 05:27:33:178 IBC: detected frame entitled: Starting application...; event=Closed
2024-03-21 05:27:33:179 IBC: detected frame entitled: Starting application...; event=Lost focus
2024-03-21 05:27:33:179 IBC: detected frame entitled: Starting application...; event=Deactivated
2024-03-21 05:27:33:179 IBC: detected frame entitled: U95482 Interactive Brokers; event=Activated
2024-03-21 05:27:33:181 IBC: detected frame entitled: U95482 Interactive Brokers; event=Focused

In this, the main window loses focus and is deactivated, a 'Connecting to server...' dialog is shown and changes its title to 'Starting application,,,' (which is TWS automatically getting going again after the reset), and then the main window is diaplayed again.

But your log looks like this at that point:

2024-03-21 00:23:06:613 IBC: detected frame entitled: Connecting to server...; event=Opened
2024-03-21 00:23:06:613 IBC: detected frame entitled: Connecting to server...; event=Activated
2024-03-21 00:23:06:672 IBC: detected frame entitled: Connecting to server...; event=Focused
2024-03-21 00:24:45:632 IBC: detected dialog entitled: Re-login is required; event=Opened
2024-03-21 00:24:45:633 IBC: Re-login to session
2024-03-21 00:24:45:633 IBC: Click button: Re-login
2024-03-21 00:24:45:710 IBC: detected frame entitled: Attempt 7: Authenticating...; event=Lost focus
2024-03-21 00:24:45:710 IBC: detected dialog entitled: Re-login is required; event=Closed
2024-03-21 00:24:45:711 IBC: detected frame entitled: Attempt 7: Authenticating...; event=Focused
2024-03-21 00:24:45:883 IBC: detected frame entitled: ; event=Lost focus
2024-03-21 00:24:45:883 IBC: detected frame entitled: ; event=Deactivated
2024-03-21 00:24:45:898 IBC: detected dialog entitled: Unrecognized Username or Password; event=Opened
2024-03-21 00:24:45:907 IBC: detected dialog entitled: Unrecognized Username or Password; event=Activated
2024-03-21 00:24:45:908 IBC: detected dialog entitled: Unrecognized Username or Password; event=Focused

So my TWS seems to be automatically continuing the session, whereas yours is demanding re-login.

Why the difference? As far as I can see, the only differences between your setup and mine are these:

• you have the Daily Lineup displayed earlier in the session, but that was closed hours before, so I doubt that has any influence
• you have MinimizeMainWindow=yes in config.ini; again I do't see why that should have any effect

Can I suggest you turn off the Daily Lineup, and remove the minimize setting, just to see if it makes any difference.

Also when you see the 'Unrecognized Username or Password' dialog, if you click the button does it redisplay the login dialog.? If so, if you re-enter your username and password, will it then login again correctly? I'm not averse to trying to make IBC re-enter the username and password and click Login in these circumstances, but I don't understand why it apparently logs you off, but for practically every other IBC user it just sails through with no problem.

So I have one python script that is requesting portfolio information and positions that is always connected to the API. I thought that might be a problem but I wasn't sure why that would effect the username and password. I can try tonight to put it back into paper mode without that script running.

I actually did get rid of the daily lineup since those logs, mainly because it was just annoying, but the username issue continued. But I will get rid of the minimize setting to see if that helps

When I click the "Unknown Username" dialog, I have about a split second before it reappears, so I'm not even sure if having IBC try to enter the information again would even do anything.

I'll get kill my python portfolio script tonight and see if that does anything. Hopefully that fixes it, although I'll feel bad if you spent hours for my stupid script

[longunmin]

So I ran it ovn without my portfolio information script running and I didn't incur any "Unknown" issues. So maybe it has something to do with IBKR's internal reset and trying to be connected to the API at the same time. Either way, I can just kill the script before midnight and start it again 30 minutes later. I'll keep an eye on it to see if I get it again, but seems like its the scripts issue (so user error by proxy!). Strange that it doesn't occur in live mode.

Edit: Not sure if you wanted to see the logs for any reason, but let me know if you do

[rlktradewright]

Ah thanks! I was just in the middle of replying to you when your post arrived.

TLDR: please can you tell me if you're using ib_insync? If you are, I'll need to look at the ib_insync code to see how it handles the nightly reset.

I suspect it isn't caused by your script itself. My trading platform always keeps the API connections over the reset period, without any problem. Here's an extract from this morning's logfile of one of my data collector programs (in the UK the nightly reset disconnect always happens around 05:27):

2024-03-25 05:27:21.922 S [IBAPIV100.BufferedReader:EndLogMessage] IN: Msg id (ERRORMESSAGE)=4;Version=2;Id=-1;Error code=1100;Error msg=Connectivity between IB and Trader Workstation has been lost.;Advanced order reject=;
2024-03-25 05:27:21.922 N [IBAPIV100.InputMessageParser:getErrorMsg] Connection to IB has been lost
.
.
. several more error 1100 messages here omitted for clarity
.
.
2024-03-25 05:27:36.298 S [IBAPIV100.BufferedReader:EndLogMessage] IN: Msg id (ERRORMESSAGE)=4;Version=2;Id=-1;Error code=1102;Error msg=Connectivity between IB and Trader Workstation has been restored - data maintained. All data farms are connected: usfuture; eufarm; cashfarm; eufarmnj; usfarm; euhmds; ushmds; secdefeu.;Advanced order reject=;
2024-03-25 05:27:36.299 N [IBAPIV100.InputMessageParser:getErrorMsg] Connection to IB recovered: no loss of data

You can see that at 05:27:21 an Error 1100 was received notifying that TWS had lost its connection to the IB account server. At this point my code prevents any orders being placed, and queues any new market data requests, but does nothing else except logging the several additional 1100 messages that arrive. Then at 05:27:36, an error 1102 arriives, which tells my code that normal API service is resumed and no market data has been lost: my code then re-enables order placing and also starts any new market data requests that have been queued. Then everything just runs as normal. So about 15 seconds of disconnection and then everything is fine. Note in particular that it wasn't necessary for my code to reconnect to the API (in fact it didn't really have to do anything at all

So I'm fairly certain that it isn't something your script is doing that is causing ths problem.

Do you use ib_insync? I don't use it myself but I have occasionally looked at the code, and I believe it uses IBC internally if TWS isn't already running? If that's the case, then it's possible that it does something troublesome when the nightly reset disconnection happens. However if that were the case, I'd expect a lot more reports of this issue, given the large number of ib_insync users, but it may just be that hardly anyone keeps their Python scripts running overnight.

If you start IBC yourself, before running your script, then ib_insync will do nothing regarding IBC when your script connects, because the connection succeeds so TWS must already be up and running. But when it gets the error 1100 or 1102 , maybe ib_insync for some reason launches IBC: this would then lead to precisely what I've been saying all along: another instance of TWS starts, and it all goes wrong. That's why I'll need to look at the ib_insync code.

[longunmin]

I am not using ib_insync, but ibapi (although maybe I should investigate ib_insync, since my script doesn't operate exactly how I want, but that's for another day).

Admittedly, I am kludging my way through the code (I'm sure an experienced eye would cringe at it), so entirely likely I'm doing something that I shouldn't which is causing the conflict. Happy to post that script, if you think that would be of value in finding the issue

[rlktradewright]

Ok, in that case I think it would be worthwhile posting the script, either here or email it direct to me at rlking at aultan.com.

[longunmin]

No problem, here if my basic portfolio information script:

from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
import paho.mqtt.client as mqtt
import threading
import time

class IBapi(EWrapper, EClient):
    def __init__(self):
        EClient.__init__(self, self)

class IBKRClient(EClient):
    def __init__(self, wrapper):
        EClient.__init__(self, wrapper)

class IBKRWrapper(EWrapper):
    def __init__(self):
        EWrapper.__init__(self)

    def error(self, reqId, errorCode, errorString):
        print("Error:", reqId, errorCode, errorString)

    def updateAccountValue(self, key: str, val: str, currency: str, accountName: str):
        if key == "NetLiquidation":  # Check if the updated value is Net Liquidation
            self.net_liquidation_value = float(val)  # Update the net liquidation value variable
            print("Net Liquidation Value updated:", self.net_liquidation_value)

            client.publish("bot/avg_cost/", str(self.net_liquidation_value))

   def updatePortfolio(self, contract: Contract, position: float, marketPrice: float,
                        marketValue: float, averageCost: float, unrealizedPNL: float,
                        realizedPNL: float, accountName: str):
        client.publish("bot/position/", str(position))
        client.publish("bot/realized/", str(realizedPNL))
        client.publish("bot/unrealized/", str(unrealizedPNL))
        client.publish("bot/status/", "Connected")

        print("Portfolio Update - Contract:", contract.symbol, "Position:", position, "Market Price:", marketPrice,
              "Market Value:", marketValue, "Unrealized P&L:", unrealizedPNL,
              "Realized P&L:", realizedPNL, "Account:", accountName)

def on_connect(client, userdata, flags, rc):
    print("Connected to MQTT broker with result code "+str(rc))

def on_disconnect(client, userdata, rc):
    print("Disconnected from MQTT broker with result code "+str(rc))

def run_ibkr():
    timeout = time.time() + 900  # Set timeout for 5 minutes
    while True:
        try:
            if time.time() > timeout:
                print("Reconnection timeout reached. Stopping reconnection attempts.")
                client.publish("bot/status/", "Disconnected")
                break

            wrapper = IBKRWrapper()
            ibkr_client = IBKRClient(wrapper)
            ibkr_client.connect("xxx.xxx.xx.xx", xxxx, clientId=0)  
            ibkr_client.reqAccountUpdates(subscribe=True, acctCode="X")
            ibkr_client.run()
        except Exception as e:
            print("Exception occurred:", e)
            time.sleep(10)  # Retry connection after 10 seconds

def publish_updates():
    global client
    client.loop_start()  
    while True:
        time.sleep(30)  
        client.publish("bot/status/", "Connected")

def main():
    global client
    client = mqtt.Client()
    client.on_connect = on_connect
    client.on_disconnect = on_disconnect
    client.connect("xxx.xxx.xx.xx", xxxx, 60)  

    ibkr_thread = threading.Thread(target=run_ibkr)
    ibkr_thread.start()

    publish_thread = threading.Thread(target=publish_updates)
    publish_thread.start()

if __name__ == "__main__":
    main()

[rlktradewright]

It's taken me a while to get my head around this script, but i think I understand what it's trying to do now.

The fact that you're using both TWSAPI client and the MQTT client in the same script, along with calls tr sleep() - which are always a 'code smell' in TWS API - rings alarm bells.

Both clients run their own message loops, and in this kind of architecture you definitely don't want to hold up the message loops by using sleep() - because the message loop can't do its job if the thread is blocking.

Whether this could be the cause of your problem I don't know. I can imagine that if the various threads are blocking at the time TWS wants to do its reset, things might go a bit awry, but i can't speculate on the details.

I suggest you try to rework the script so that there are no calls to sleep(): for example if you want to do something after n seconds, don't sleep the thread for that period, start a timer and when it expires and calls its expiry handler, then do your 'something'.

[longunmin]

Thank you! I will try and rework the script without sleep() [sorry again for the quality of the script, I warned you it would be gross!]. I have gotten the error the passed two nights, but in both instances the script wasn't running. However, I did have a job set up to initiate the script after the TWS internal reset. I'll kill that job tonight and see if the error occurs. If it does, a good indication it isn't my clunky script causing the problem. If no error, well, walks like a duck.

I'll report back if it occurs again without my script running. Thank you again

[longunmin]

So I ran it the last few nights without the script running and i continue to get the error, so while clunky and inefficient, the script doesn't appear to be the issue

Please let me know if you would like to see the logs from the last few nights (or if there is something else I should try, like just reinstalling everything). Thank you!

[rlktradewright]

I guess I'm not surprised to hear that.

Here are a few more suggestions:

• Set ReloginAfterSecondFactorAuthenticationTimeout=yes in config.ini. I don't think this will actually change anything as regards your problem, but it does improve the handling of some things that are shown in your logs

• Make absolutely sure you don't have any cron jobs running related to IBC (so start it IBC manually)

• Don't fiddle with anything during the IBC startup and login sequence (ie no keypresses or mouse clicks)

• Do a run with LogStructureScope=all and LogStructureWhen=open in config.ini. This will record details of the structure of each dialog displayed by TWS to the IBC logfile. It's just possible that this might provide additional info that will help. It will make the IBC logfile large, so don't forget to set it back to normal once you've captured the logfile and attached it here.

• Make sure you have installed the latest issue of TWS 10.28 (or whatever version you're using). 10.28.1d I think.

• Something worth a try is just to run TWS manually (ie without using IBC at all), and see if it gets past the nightly reset ok, without 'Re-login is required' messages

• It might actually be worth uninstalling TWS, then renaming your current IBC and Jts installation folders, and then reinstalling everything. I had a problem once where TWS had remembered something in its huge settings file that was causing problems. Once I uninstalled, wiped out the Jts folder structure, and reinstalled, it was ok.

• Another thing worth doing would be to zip your entire Jts and IBC folder stuctures, including the scripts (but you can leave out the logfiles!), and put it somewhere where i can access it. I can then load the whole thing on to my Ubuntu VM and see if I have the same problem with my account (so make sure to remove your user id and password).

[longunmin]

Thank you for all these suggestions, and I will start trying them one by one, to see if any of them fix the issue on their own.

However, before I start going through that list, I think I have some new data points, which may help. So I may have previously mentioned, or least indicated by the logs, that I moved the Ibkr auto-start time to 6:45PM EST (this was because I had a few instances ovn, where IBC wasn't auto logging in, and I wanted to see what was occurring). Smash cut, IBC works fine and is auto logging in at the 6:45PM time, but now we are getting the aforementioned issue when IBKR has their internal reset.

However, the last few nights after the IBC auto login at 6:45, I have been manually killing IBC and rerunning ./twsstart.sh (I stumbled on to this "solution" because I was working on getting rid of sleep() in my scripts per your suggestion). Whenever I kill IBC after the 6:45 restart, I never get an occurrence of the failed login during the IBKR internal reset.

So the above would seem to indicate that you are correct from the other thread where you said IBC/IBKR is creating multiple instances (and my manually killing one after 6:45 is a "solution"), but I have no idea why that would be. As this issue seems to be localized to "paper mode", I'm fine to use this as temporary solution, as this seems to be an edge case issue (and, in all probabilities, one caused by user error somewhere along the line). I thought maybe I could run a cron job (sidenote: you had asked previously, and no, I don't have any other cron jobs currently) to kill IBC and restart it, but the stop.sh script appears to be empty, so I wasn't sure how to close it all down outside of the GUI.

Again, happy to post logs (and I will go through your previously mentioned steps), but I just wanted to share my purely observational data, in case that triggered any lightbulbs for you.

[rlktradewright]

First, the stop.sh scripts is available in from GitHub per these links (you’ll need both scripts):

https://github.com/IbcAlpha/IBC/blob/master/resources/stop.sh https://github.com/IbcAlpha/IBC/blob/master/resources/commandsend.sh

These scripts will be included in the zip for the next release.

I’m not sure I really understand what you’ve written about killing and then manually restarting it. Or rather, I understand what you’re saying but not why it should make any difference.

Can you please check something for me? The TWS installer creates a monster script called tws in the installation folder (for example ~\Jts\1028\tws). This is what is used to start TWS manually (ie not running under IBC). When TWS is auto-restarting, it also calls this script to create another instance of TWS immediately before the original one exits. However, IBC doesn’t call this script when running TWS: instead it loads the TWS jar files directly into memory. So for auto-restart to run properly when using IBC, we have to prevent the existing TWS instance creating a new instance by calling this script, because if it manages to do this the new instance won’t be running under IBC. To prevent this happening, the twsstart.sh and gatewaystart.sh scripts rename the script to tws1: then when TWS tries to start the new instance using the tws script, it can’t find the script, so it just exits. Instead the IBC script creates the new TWS instance in exactly the same way as it created the original one .

Now, if for some reason the tws script is not successfully renamed to tws1, then the original TWS instance will create a new instance, and so will the IBC script, and this will lead to the problems you’ve been experiencing.

So please can you check whether this tws script has been renamed to tws1? If it hasn’t, then this is the cause of all your problems, and we just have to find out why the rename is not succeeding. It may be that for some reason the script has a restrictive permission on it, in which case just set the permissions so that the user that IBC/TWS runs under has write access to the directory that contains the script.

I haven’t mentioned this before because it seemed very unlikely that this would be the case. But worth checking out.

[longunmin]

Yep there is a tws1 under the /Jts/1028

[rlktradewright]

Ok, noted.

I think the best thing for you to do is to ship the whole lot over to me so I can run with your exact configuration of IBC and TWS (except for the user accoiunt), as suggested in my post on 31 March.

Otherwise there doesn't seem to be much else I can do. All the evidence is that IBC is behving exactly as it should, but for some reason TWS is behaving differently at and after the midnight reset period, if it is an auto-restarted instance.

[longunmin]

Ok, noted.

I think the best thing for you to do is to ship the whole lot over to me so I can run with your exact configuration of IBC and TWS (except for the user accoiunt), as suggested in my post on 31 March.

Otherwise there doesn't seem to be much else I can do. All the evidence is that IBC is behving exactly as it should, but for some reason TWS is behaving differently at and after the midnight reset period, if it is an auto-restarted instance.

How do I use the stop.sh script, as I updated the commendsend.sh script with server ip and port, but when I run ./stop.sh I get the following output in the command line

Trying 192.xxx.xx.xx...
Connected to 192.xxx.xx.xx.
Escape character is '^]'.
Connection closed by foreign host.

However, neither IBC, nor TWS are closed and the API connection remains. I only ask because, closing down IBC/TWS post restart seems to fix the issue. And as this is an edge case, only relevant to paper mode, I don't want to waste your time chasing ghosts in my machine, if there is a quasi-solution (i.e. just have a job run stop.sh post restart, and then rerun twsstart.sh)

[rlktradewright]

Make sure CommandServerPort setting in config.ini is set to the same port specified in comandsend.sh.

You don't actually need to use stop.sh at all. Just set the auto-logoff time in TWS instead of auto-restart. Then just rerun twsstart.sh once TWS has closed.

I hear what you say about this being an edge case, but having spent such a lot of time trying to understand what's going on here I'll be a little disappointed if we don't get there. I'm happy to try and run your configuration: if i can understand the cause, I might be able to do something to prevent anyone else falling into this hole. It shouldn't take long to zip it all up, the only question being whether you have anywhere big enough to put it that I can access it from.

[longunmin]

Make sure CommandServerPort setting in config.ini is set to the same port specified in comandsend.sh.

You don't actually need to use stop.sh at all. Just set the auto-logoff time in TWS instead of auto-restart. Then just rerun twsstart.sh once TWS has closed.

I hear what you say about this being an edge case, but having spent such a lot of time trying to understand what's going on here I'll be a little disappointed if we don't get there. I'm happy to try and run your configuration: if i can understand the cause, I might be able to do something to prevent anyone else falling into this hole. It shouldn't take long to zip it all up, the only question being whether you have anywhere big enough to put it that I can access it from.

Oh good to know re: the autologoff, I will try that tonight and see how it works.

What files would I need to zip for you? /opt/ibc & /$USER/ibc and /$USER/Jts?

[rlktradewright]

Yes, I think that should be enough. You can exclude other TWS versions you may have installed (eg /$User/Jts/1019), ditto any Gateway installations (eg /$User/Jts/ibgateway/1028) to reduce the size.

This will be a sizeable upload, about 210MBytes on my Ubuntu system, so if this would be problematic for your internet connection then just leave it for now.

[longunmin]

OK I will try and zip it tonight (schedule allowing)

[longunmin]

Yes, I think that should be enough. You can exclude other TWS versions you may have installed (eg /$User/Jts/1019), ditto any Gateway installations (eg /$User/Jts/ibgateway/1028) to reduce the size.

This will be a sizeable upload, about 210MBytes on my Ubuntu system, so if this would be problematic for your internet connection then just leave it for now.

Sorry for the delay in getting these to you, I was away from my setup all week. Yeah and I couldn't get the JTS as it was too big

ibc.tar.gz opt.ibc.tar.gz

[rlktradewright]

Sorry for the delay in responding to this.

Thanks for sending those files, but in fact they don't contain anything I didn't already know from the logfiles.

I was hoping to be able run with your exact setup but I would have needed the Jts files, and in any case it wouldn't have worked because the settings file is encrypted and I wouldn't be able to read it. Silly me...

I've been reviewing some of the IBC code, and I've noticed something that might have a bearing on this. I'm going to modify this code, and hopefully I'll offer a new beta version within the next day or two. No guarantee this will fix it, and it's not actually something I can test myself (because I never encounter this issue odd behaviour at the nightly reset), but it's worth a try.

[longunmin]

Sorry for the delay in responding to this.

Thanks for sending those files, but in fact they don't contain anything I didn't already know from the logfiles.

I was hoping to be able run with your exact setup but I would have needed the Jts files, and in any case it wouldn't have worked because the settings file is encrypted and I wouldn't be able to read it. Silly me...

I've been reviewing some of the IBC code, and I've noticed something that might have a bearing on this. I'm going to modify this code, and hopefully I'll offer a new beta version within the next day or two. No guarantee this will fix it, and it's not actually something I can test myself (because I never encounter this issue odd behaviour at the nightly reset), but it's worth a try.

No problem! I know you have a lot of other stuff on your plate. OK, I will be happy to test it out whenever you have a beta available. My "workaround" has been very stable, for anyone who comes across this issue. I haven't had any login issue since I set TWS to no auto-restart, and just have IBC restart the session via a cron job once TWS logs off. Doesn't explain what is going on, of course, but offers a stop gap solution for anyone who may come across this issue

[rlktradewright]

Here's a link to the new beta version of IBC v3.19.0-beta.1:

https://1drv.ms/f/s!AlqfLEOWDJ9Zh-klxayDSKUDSpj0Ow?e=LUbIce

For both Windows and Linux/macOS users, you'll need to extract the updated IBC.jar and version files to your IBC folder.

For Windows users only, you should also extract the StartIBC.bat file to the scripts subfolder in your IBC folder. And also extract the config.ini file, carrying forward your settings to the new version.

You may notice that there is a new ColdRestartTime setting in config.ini. I haven't yet decided whether to keep this feature going forward with IBC, so you can use it if you want but I may remove it (though if people find it useful I'll probably keep it). However note that this feature doesn't work properly on Linux/macOS as the scripts need some updates to support it: I'll do these if I decide to keep the feature.

[longunmin]

Here's a link to the new beta version of IBC v3.19.0-beta.1:

https://1drv.ms/f/s!AlqfLEOWDJ9Zh-klxayDSKUDSpj0Ow?e=LUbIce

For both Windows and Linux/macOS users, you'll need to extract the updated IBC.jar and version files to your IBC folder.

For Windows users only, you should also extract the StartIBC.bat file to the scripts subfolder in your IBC folder. And also extract the config.ini file, carrying forward your settings to the new version.

You may notice that there is a new ColdRestartTime setting in config.ini. I haven't yet decided whether to keep this feature going forward with IBC, so you can use it if you want but I may remove it (though if people find it useful I'll probably keep it). However note that this feature doesn't work properly on Linux/macOS as the scripts need some updates to support it: I'll do these if I decide to keep the feature.

I ran this for a few days and didn't have any issues. I switched back to stable version, just because I prefer being on a main branch. But i just wanted to circle back on this for you. Sorry took so long to follow up!