ib.fills() returns incorrect results for partially filled orders for multi-leg option COMBO trades

[stenri]

I developed a script TradeLogIB which uses ib_insync to download a list of option trades from IB and save them to a .csv file. Basically what this script does is:

        for fill in ib.fills():
            if fill.contract.secType != 'OPT':
                continue
            ....

Very compact and elegant code thanks to ib_insync. What I noticed TradeLogIB script does not save trades for a partically filled multi-leg option orders. I have to cancel an order or wait till it is completely filled, and only then trades from this order can be dumped using my script.

In contract version of TradeLogIB that worked with Python 2.7 + IbPy package works quite well, and is able to dump trades for a partially filled orders. So I know something is wrong with ib.fills() in ib_insync.

Here are the results of my research. When multi-leg option order is partially filled, ib.fills() returns something like this:

Notice a list of Fill() structures containing the same Contract() with the same conId and secType='BAG' indicating this is a contract for a multi-leg COMBO order.

And after the multi-leg option order is filled, ib.fills() finally returns the following:

This is a correct return result, what I expect it to return in both cases. Notice the first Fill() contains a Contract() with secType='BAG' (COMBO order). And second and third lines contains a Contract() with secType='OPT' (fills for individual option legs for a COMBO order above).

At this point I knew something was wrong with the Contract() field in the Fill() structure when order was in a partially filled state.

Now, let's take a look at wrapper.py at execDetails() implementation in ib_insync:

    @iswrapper
    def execDetails(self, reqId, contract, execution):
        # must handle both live fills and responses to reqExecutions
        key = (execution.clientId, execution.orderId)             # <--------- (1)
        trade = self.trades.get(key)
        if trade:
            contract = trade.contract                             # <--------- (2) BUGBUG: 
        else:
            contract = Contract(**contract.__dict__)
        execId = execution.execId
        execution = Execution(**execution.__dict__)
        fill = Fill(contract, execution, CommissionReport(), self.lastTime)

execDetails() merges self.trades.contract (opened orders) with the contract data that is passed as a parameter (see linke marked (2) BUGBUG:). And it uses a key consisting from clientId and orderId (see line marked (1) in the code above).

The problem is that multi-leg option COMBO orders contain several Fill()'s, each one with it's own Contract(). And all these Contract()'s have the same orderId and clientId as well. Some of the contracts are secType='BAG', others are secType='OPT' (see screenshot above). But orderId is the same as all these Contract()'s and Fill()'s belong to the same COMBO order.

So, when execDetails() decides not to use a contract passed as a function argument, but instead take the trade.contract based on key==(execution.clientId, execution.orderId):

        if trade:
            contract = trade.contract                             # <--------- (2) BUGBUG: 

It erroneously substitutes real Contract(secType='OPT') with a Contract(secType='BAG') from self.trades structure. And that causes problems.

Сonclusion: execDetails() can not use clientId / orderId pair as a key to determine a Contract() as it does not work correctly for a multi-leg option orders.

As a proof of concept I developed a fix to verify it solves the issue. If I rewrite the execDetails() code to throw out the offensive lines:

    @iswrapper
    def execDetails(self, reqId, contract, execution):
        # must handle both live fills and responses to reqExecutions
        key = (execution.clientId, execution.orderId)
        trade = self.trades.get(key)
        contract = Contract(**contract.__dict__)
        execId = execution.execId
        execution = Execution(**execution.__dict__)
        fill = Fill(contract, execution, CommissionReport(), self.lastTime)

TradeLogIB script starts to work as expected and correctly dumps partially filled multi-leg orders.

I do not know if this change breaks anything else, as it is not clear to me what the original intention was in replacing the contract in the Fill() structure. So, I am going to submit my patch as a pull request, and it is up to the ib_insync developer to decide how to fix this correctly.

P.S. I've also considered other methods to fix this. For example to use the key that contains "clientId, orderId, contract Id":

 key = (execution.clientId, execution.orderId, contract.conId)
```But I found that at least in one place in orderStatus() routine there is no access to a contract at all. And orderStatus() calculates a key.

[erdewit]

Thank you very much for the analysis Stan. I have applied the patch and modified it a little to reuse an existing contract object, if possible. The intention is to have the exact same contract object (by identity) for the order and all its fills.

Cheers, Ewald

[erdewit]

Btw for your project the flex webservice or FlexReport might be of interest.