ByPass Order Precautions for API?

[AlphaGit]

Hi there!

I've recently upgraded IB TWS to its 10.19 version, along with IBC to 3.17.

However, in my particular case, I'm sending orders by manually specifying their exchange, say, specifically ISLAND.

I'm being hit with this error message:

Error 10311, reqId 9: This order will be directly routed to ISLAND. Direct routed orders may result in higher trade fees. Restriction is specified in Precautionary Settings of Global Configuration/API.

Now, there are two settings in IB that will help to this particular issue:

They are:

• Bypass Order Precautions for API Orders
• Bypass Redirect Order warning for Stock API Orders

I remember that old versions of IBC used to have a ForceTwsApiBypassPrecautions setting, which is not present in this one anymore.

Is there any reason for it not to be there anymore? If not, I am even willing to submit code to add it again. I imagined there must be a reason though -- hence this ticket asking about it.

As an additional piece of info, TWS 9.83 will allow me to submit orders to ISLAND regardless of this setting value.

Another piece of info: these settings do not seem to be persisted in the server configuration, even if the settings say to retrieve settings from there.

Regards!

[rlktradewright]

Which old IBC version? I don't recollect there ever being such a setting.

What TWS 983 does is irrelevant. Are you really still using this old version?

Can you please clarify what you mean by 'these settings do not seem to be persisted in the server configuration'. Which settings? All the API - Precautions? Some of them? Which TWS version? Etc.

I will consider accepting a Pull Request for implementing a new setting in 'config.ini` - in fact I'm rather surprised that users who run IBC in Docker haven't asked for something like this, but perhaps nearly everyone uses SMART routing and aren't affected by this.

But if you want to do this, please post a draft of the proposed update to config.ini for discussion before implementing it.

[AlphaGit]

Which old IBC version? I don't recollect there ever being such a setting.

If I recall correctly, this repo was heavily inspired by https://github.com/ib-controller/ib-controller -- this is what I meant by the old IBC. If I'm mistaken in my assumption, I apologize.

What TWS 983 does is irrelevant. Are you really still using this old version?

Yes. :( However, this might allow me to upgrade.

Can you please clarify what you mean by 'these settings do not seem to be persisted in the server configuration'. Which settings? All the API - Precautions? Some of them? Which TWS version? Etc.

All of the API precautions do not seem to be persisted in IB's server. They seem to get reset each time that the application starts. The way I tested it was VNC'ing into the automated instance and manually updating the settings. Upon closing and re-opening it, the settings were back to their default (do not bypass).

I will consider accepting a Pull Request for implementing a new setting in 'config.ini` - in fact I'm rather surprised that users who run IBC in Docker haven't asked for something like this, but perhaps nearly everyone uses SMART routing and aren't affected by this.

I assumed this was the case, too.

I came upon this "lost setting" by searching around for similar issues, and I found that people used to set that setting to fix it. However, going on the original repo does not show that setting.

I came across this ticket and associated PR:

• https://github.com/ib-controller/ib-controller/issues/42
• https://github.com/ib-controller/ib-controller/pull/43

Those introduce these settings (along with some others), but were not looked with good eyes since they didn't want to turn IBC into a "TWS auto-configuration tool".

If you're okay with the approach, I'll submit one following the same approach.

But if you want to do this, please post a draft of the proposed update to config.ini for discussion before implementing it.

Sure, this is what I had in mind:

(Right after ReadOnlyApi)

# API Precautions
# For all of these, the accepted values are:
# - 'enable' (for backward compatibility) / 'yes': Checks the box
# - 'disable' (for backward compatibility) / 'no': Unchecks the box (no bypassing)
# - 'manual' (default): No changes done by IBC

# Bypass API Precautions
ForceTwsApiBypassPrecautions=

# Bypass Bond Warning For API Orders
ForceTwsApiBypassBondWarning=

# Bypass negative yield to worst confirmation for API Orders
ForceTwsApiBypassNegativeYield=

# Bypass Called Bond warning for API Orders
ForceTwsApiBypassCalledBond=

# Bypass "same action pair trade" warning for API Orders
ForceTwsApiBypassSameActionPairTrade=

# Bypass price-based volatility risk warning for API Orders
ForceTwsApiBypassPriceBasedVolatilityRisk=

# Bypass US Stocks market data in shares warning for API Orders
ForceTwsApiBypassStocksInShares=

# Bypass Redirect Order warning for Stock API Orders
ForceTwsApiBypassRedirecOrder=

# Bypass No Overfill Protection precaution for destinations where implied natively
ForceTwsApiBypassNoOverfillProtection=

A few notes on the names:

• They are quite lengthy. I just wanted to keep backwards compatibility with that old setting I found, but I'd personally go for more concise names.
• I added them all for the sake of completion, but my case only needs the first one (Order Precautions) and the Redirect Order warning.

Let me know what you think!

[rlktradewright]

Thanks. Sorry for the delay. I do have some comments, but I'm going to be out for most of the day, so I'll try to respond this evening.

[rlktradewright]

IBC was indeed forked from IBController. I was the principal developer of IBController, as I am for IBC. The need for the fork was the result of an unpleasant situation that arose, which I don't intend to describe here, but which resulted in IBController's development and support being seriously curtailed and me unable to influence it.

Thanks for digging up that IBController PR. I had completely forgotten about that. Its rejection confirms the fact that there has never been an 'official' implementation of this order precautions override setting.

The misgivings I expressed back then about not wanting to turn IBController into a TWS configuration tool apply equally to IBC. However, the issue was forced to some extent by IBKR introducing the 'read-only API' setting, which by default is true (ie a fresh installation of TWS or Gateway has 'read-only API' checked out-of-the box). For Gateway this is ridiculous - I suspect hardly any Gateway users use this mode. The effect of this was that the common approach to Docker images of running a clean installation of Gateway was imnmediately broken, because you couldn't do anything useful with it.

So at that stage I introduced the ReadOnlyAPI setting in config.ini, much against my will, because I knew it would be the thin end of the wedge, and indeed since then it's been necessary to add several more such settings which can only be set via the GUI and don't have sensible defaults.

That's the background to why I'm reluctantly open to allowing the settings your propose.

Ok, that's enough preaching!...

Next, I'm very puzzled by what you say regarding these settings not being persisted between sessions. I've never seen such a thing, regardless of whether I use 'store settings on server'. I can only suggest you take this up with IBKR - perhaps the TWS logfile contains something that would help them work out what's happening.

Regarding the proposed settings, I think the names are too unwieldy. I'd prefer them just to be the same as on the TWS configuration page, but without unnecessary verbiage, eg BypassOrderPrecautions, BypassSameActionPairTradeWarning. No need to have any similarity to the related IBController setting you discovered.

The comment should point out that these settings are really only for use where the GUI is inaccessible, or where settings are not persisted. See other such settings for examples,

For the implementation, a single ConfigureApiPrecautionsTask class that can handle all these new settings will be fine. Note that the mechanism is no longer as it was in IBController, so refer to some of the other configuration tasks for a jumpstart.

And bear in mind that some TWSs that are still in use don't support all these settings, so not finding one of the checkboxes should be logged but not considered an error.

I think that's it for now. Good luck!

[AlphaGit]

Sounds good! Thanks for all the explanation, I think it clears up a lot. I'll take your suggestion and investigate a bit more on the server persistence, and update back with my findings. As I have a currently working version, my priorities shifted otherwise but I'd still like to come back to this -- working with a version from 2021 is a time bomb.

[rlktradewright]

Ok just to clarify, are you offering to do this work yourself, or would you prefer me to do it? It wouldn't be a big task for me, so I'm happy to do it.

[AlphaGit]

I have not set up a Java project in a while so it might take me a bit to get running. I was offering to do it myself, but if you had the time and you think it'd be easier, I'd really appreciate you doing it.

If you can't / don't want to, that's absolutely okay and I'll get to it soon.

[rlktradewright]

OK, leave it with me. I'll update here soon.

[rlktradewright]

This has been done in Release 3.18.0, issued today.

The setting names are different from what you proposed, as are the permitted values, but the notes in config.ini give the details.

[AlphaGit]

I just tried it out with our current setup and it works like a charm! Thank you so so much for your help on this!