weird expiry date format for GASOIL, EURIBOR-ICE, ROBUSTA

[bug-or-feature]

I'm seeing some unexpectedly formatted expiry dates for some instruments from IB. Unfortunately I can't use a debugger with the instance, so I added this code at around line 125 of sysbrokers/IB/ib_futures_contract_data.py:

if len(expiry_date) > 8:
    log.info(f"Attempting to convert expiry date: {expiry_date}")

That results in log messages like:

2023-08-22 13:55:31 INFO stack_handler {'instrument_code': 'EURIBOR-ICE', 'contract_date': '20250900'} Attempting to convert expiry date: '20250915 10:00:00 GB'
2023-08-22 13:55:33 INFO stack_handler {'instrument_code': 'GASOIL', 'contract_date': '20231100'} Attempting to convert expiry date: '20231110 12:00:00 GB'
2023-08-22 13:55:38 INFO stack_handler {'instrument_code': 'ROBUSTA', 'contract_date': '20231100'} Attempting to convert expiry date: '20231124 12:30:00 GB'

I first saw this last week, but I was busy with other stuff and couldn't deal with it properly. I just hacked expiryDate.from_str() to only consider the first 8 characters of expiry strings. But today I noticed a contract order on the stack with waaaay too many children. It turned out that there was a contract order to buy some EURIBOR-ICE/20250900; The broker orders were getting cancelled, with log messages like:

2023-08-22 09:30:24 ERROR ib_insync.wrapper Error 478, reqId 747475: Parameters in request conflicts with contract parameters received by contract id: requested expiry 20250915 10:00:00 GB, in contract 20250915;
2023-08-22 09:30:24 WARNING ib_insync.wrapper Canceled order: Trade(contract=Contract(secType='FUT', conId=383665193, symbol='I', lastTradeDateOrContractMonth='20250915 10:00:00 GB', multiplier='2500', exchange='ICEEU', currency='EUR', localSymbol='IU5', tradingClass='I'), order=LimitOrder(orderId=747475, clientId=xxx, action='BUY', totalQuantity=1, lmtPrice=96.93, account='xxx'), orderStatus=OrderStatus(orderId=747475, status='Cancelled', filled=0.0, remaining=0.0, avgFillPrice=0.0, permId=0, parentId=0, lastFillPrice=0.0, clientId=0, whyHeld='', mktCapPrice=0.0), fills=[], log=[TradeLogEntry(time=datetime.datetime(2023, 8, 22, 8, 30, 24, 530754, tzinfo=datetime.timezone.utc), status='PendingSubmit', message='', errorCode=0), TradeLogEntry(time=datetime.datetime(2023, 8, 22, 8, 30, 24, 573307, tzinfo=datetime.timezone.utc), status='Cancelled', message='Error 478, reqId 747475: Parameters in request conflicts with contract parameters received by contract id: requested expiry 20250915 10:00:00 GB, in contract 20250915;', errorCode=478)], advancedError='')
2023-08-22 09:30:24 DEBUG stack_handler {'component': 'ibContractPositionData'} Reqid 747475: 478 Parameters in request conflicts with contract parameters received by contract id: requested expiry 20250915 10:00:00 GB, in contract 20250915; for None
2023-08-22 09:30:24 DEBUG stack_handler {'component': 'ibFuturesInstrumentData'} Reqid 747475: 478 Parameters in request conflicts with contract parameters received by contract id: requested expiry 20250915 10:00:00 GB, in contract 20250915; for None
2023-08-22 09:30:24 DEBUG stack_handler {'component': 'ibFuturesContractData'} Reqid 747475: 478 Parameters in request conflicts with contract parameters received by contract id: requested expiry 20250915 10:00:00 GB, in contract 20250915; for None
2023-08-22 09:30:24 DEBUG stack_handler {'component': 'ibFuturesContractPriceData'} Reqid 747475: 478 Parameters in request conflicts with contract parameters received by contract id: requested expiry 20250915 10:00:00 GB, in contract 20250915; for None
2023-08-22 09:30:24 DEBUG stack_handler {'component': 'ibStaticData'} Reqid 747475: 478 Parameters in request conflicts with contract parameters received by contract id: requested expiry 20250915 10:00:00 GB, in contract 20250915; for None
2023-08-22 09:30:24 DEBUG stack_handler {'component': 'ibExecutionStackData'} Reqid 747475: 478 Parameters in request conflicts with contract parameters received by contract id: requested expiry 20250915 10:00:00 GB, in contract 20250915; for None
2023-08-22 09:30:24 WARNING stack_handler {'strategy_name': 'caleb_strategy_v3', 'instrument_code': 'EURIBOR-ICE', 'contract_order_id': 2289, 'broker_order_id': 5637} Order has been cancelled: not by algo
2023-08-22 09:30:24 DEBUG stack_handler {'type': 'run_stack_handler'} Released contract order 2289 from algo control

And then of course, a new broker order would be created, cancelled, and another, etc etc. I noticed after about 300

Anyone know what's going on with the expiry dates? Or know how to resolve the orders getting cancelled by IB?

[robcarver17]

I didn't see these yesterday, but looking at todays logs there are a whole bunch of errors that look similar:

2023-08-23 04:30:29 DEBUG stack_handler {'component': 'ibContractPositionData'} Reqid 79679: 354 Requested market data is not subscribed. for None
2023-08-23 04:30:29 DEBUG stack_handler {'component': 'ibFuturesInstrumentData'} Reqid 79679: 354 Requested market data is not subscribed. for None
2023-08-23 04:30:29 DEBUG stack_handler {'component': 'ibFuturesContractData'} Reqid 79679: 354 Requested market data is not subscribed. for None
2023-08-23 04:30:29 DEBUG stack_handler {'component': 'ibFuturesContractPriceData'} Reqid 79679: 354 Requested market data is not subscribed. for None
2023-08-23 04:30:29 ERROR ib_insync.wrapper Error 300, reqId 79679: Can't find EId with tickerId:79679
2023-08-23 04:30:29 DEBUG stack_handler {'component': 'ibContractPositionData'} Reqid 79679: 300 Can't find EId with tickerId:79679 for None
2023-08-23 04:30:29 DEBUG stack_handler {'component': 'ibFuturesInstrumentData'} Reqid 79679: 300 Can't find EId with tickerId:79679 for None
2023-08-23 04:30:29 DEBUG stack_handler {'component': 'ibFuturesContractData'} Reqid 79679: 300 Can't find EId with tickerId:79679 for None
2023-08-23 04:30:29 DEBUG stack_handler {'component': 'ibFuturesContractPriceData'} Reqid 79679: 300 Can't find EId with tickerId:79679 for None
2023-08-23 04:30:29 INFO arctic.serialization.numpy_records Index has no name, defaulting to 'index'

[robcarver17]

Perhaps an update to the IB API which I'm not seeing? What happens if you run native ibinsync?

Following checks the contract expiry for robusta

Python 3.8.5 (default, Jan 27 2021, 15:41:15) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ib_insync import *
>>> util.startLoop()
>>> 
>>> import logging
>>> # util.logToConsole(logging.DEBUG)
>>> 
>>> ib = IB()
>>> ib.connect('127.0.0.1',4001,clientId=99999)
<IB connected to 127.0.0.1:4001 clientId=99999>
>>> f = Future('D',exchange='ICEEUSOFT')
>>> c = ib.reqContractDetails(f)
>>> c[0].contract.lastTradeDateOrContractMonth
'20230925'

[bug-or-feature]

Thanks for that snippet, very useful

ContractDetails(contract=Contract(secType='FUT', conId=553444806, symbol='D', lastTradeDateOrContractMonth='20231124 12:30:00 GB', multiplier='10', exchange='ICEEUSOFT', currency='USD', localSymbol='RCX3', tradingClass='RC'), marketName='RC', minTick=1.0, orderTypes='ACTIVETIM,AD,ADJUST,ALERT,ALGO,ALLOC,AVGCOST,BASKET,BENCHPX,COND,CONDORDER,DAY,DEACT,DEACTDIS,DEACTEOD,GAT,GTC,GTD,GTT,HID,ICE,IOC,LIT,LMT,MIT,MKT,MTL,NGCOMB,NONALGO,OCA,OPENCLOSE,PEGBENCH,SCALE,SCALERST,SNAPMID,SNAPMKT,SNAPREL,STP,STPLMT,TRAIL,TRAILLIT,TRAILLMT,TRAILMIT,WHATIF', validExchanges='ICEEUSOFT', priceMagnifier=1, underConId=27655507, longName='Robusta Coffee', contractMonth='202311', industry='', category='', subcategory='', timeZoneId='GB-Eire', tradingHours='20230823:0900-20230823:1730;20230824:0900-20230824:1730;20230825:0900-20230825:1730', liquidHours='20230823:0900-20230823:1730;20230824:0900-20230824:1730;20230825:0900-20230825:1730', evRule='', evMultiplier=0, mdSizeMultiplier=1, aggGroup=2147483647, underSymbol='D', underSecType='IND', marketRuleIds='33', secIdList=[], realExpirationDate='20231124', lastTradeTime='Eire', stockType='', minSize=1.0, sizeIncrement=1.0, suggestedSizeIncrement=1.0, cusip='', ratings='', descAppend='', bondType='', couponType='', callable=False, putable=False, coupon=0, convertible=False, maturity='', issueDate='', nextOptionDate='', nextOptionType='', nextOptionPartial=False, notes='')

lastTradeDateOrContractMonth='20231124 12:30:00 GB'

So where is that date format coming from, could be a user preference or something?

[bug-or-feature]

For realExpirationDate the date format is consistent in both

[robcarver17]

What gateway version are you running? 10.23 here

[bug-or-feature]

Same 10.23

[bug-or-feature]

hmmm. I can identify EURIBOR-ICE with 202509, but not 20250900

>>> euribor = Future('I',exchange='ICEEU',lastTradeDateOrContractMonth='20250900')
>>> c = ib.reqContractDetails(euribor)
Error 200, reqId 3: Invalid value in field # 541, contract: Future(symbol='I', lastTradeDateOrContractMonth='20250900', exchange='ICEEU')
>>> euribor = Future('I',exchange='ICEEU',lastTradeDateOrContractMonth='202509')
>>> c = ib.reqContractDetails(euribor)
>>> c[0]
ContractDetails(contract=Contract(secType='FUT', conId=383665193, symbol='I', lastTradeDateOrContractMonth='20250915 10:00:00 GB', multiplier='2500', exchange='ICEEU', currency='EUR', localSymbol='IU5', tradingClass='I'), marketName='I', minTick=0.005, orderTypes='ACTIVETIM,AD,ADJUST,ALERT,ALGO,ALLOC,AVGCOST,BASKET,BENCHPX,COND,CONDORDER,DAY,DEACT,DEACTDIS,DEACTEOD,GAT,GTC,GTD,GTT,HID,ICE,IOC,LIT,LMT,MIT,MKT,MTL,NGCOMB,NONALGO,OCA,OPENCLOSE,PEGBENCH,REL,RELPCTOFS,SCALE,SCALERST,SNAPMID,SNAPMKT,SNAPREL,STP,STPLMT,TRAIL,TRAILLIT,TRAILLMT,TRAILMIT,WHATIF', validExchanges='ICEEU', priceMagnifier=1, underConId=12148668, longName='3 Month EURIBOR Interest Rate', contractMonth='202509', industry='', category='', subcategory='', timeZoneId='GB-Eire', tradingHours='20230823:0100-20230823:2100;20230824:0100-20230824:2100;20230825:0100-20230825:2100;20230826:CLOSED;20230827:CLOSED;20230828:0100-20230828:2100', liquidHours='20230823:0100-20230823:2100;20230824:0100-20230824:2100;20230825:0100-20230825:2100;20230826:CLOSED;20230827:CLOSED;20230828:0100-20230828:2100', evRule='', evMultiplier=0, mdSizeMultiplier=1, aggGroup=2147483647, underSymbol='I', underSecType='IND', marketRuleIds='57', secIdList=[], realExpirationDate='20250915', lastTradeTime='Eire', stockType='', minSize=1.0, sizeIncrement=1.0, suggestedSizeIncrement=1.0, cusip='', ratings='', descAppend='', bondType='', couponType='', callable=False, putable=False, coupon=0, convertible=False, maturity='', issueDate='', nextOptionDate='', nextOptionType='', nextOptionPartial=False, notes='')

Ignore that, it works with 20250915

[robcarver17]

Same gateway version implies it's a TWS setting or user preference problem.

[bug-or-feature]

Yep. There's a Gateway config setting Send instrument specific attributes for dual-mode API client in with possible values operator timezone, instrument timezone, UTC. instrument timezone sends the weird format, the other two use the expected. Thank you for your help

I'll wait and check trades go through before closing

[bug-or-feature]

All good, closing. Thanks again

A couple of nice to have ideas that would make dealing with this sort of thing easier in future:

1. The option to also delete child orders when deleting an order via Interactive order stack (4, 41)
2. An email warning if a broker order gets repeatedly cancelled by the broker