Fill or kill orders stuck in executable state

I have been placing FILL_OR_KILL orders through Flumine and noticed that the order stream update for setting the order status to "Execution complete" is sometimes severely delayed. A FILL_OR_KILL order is either matched instantaneously or rejected, so it should go straight from "Executable" to "Execution complete". Normally, it looks like this, and one can see that the delta between the last two updates is 52 ms:

Time                       Selection ID  Bet ID        Order status        Side      Matched    Remaining  Trade status
-----------------------  --------------  ------------  ------------------  ------  ---------  -----------  --------------
2023-08-13 10:35:25,142        55258054  None          Pending             BACK         0            2     Live
2023-08-13 10:35:25,340        55258054  316818562580  Executable          BACK         2            0     Pending
2023-08-13 10:35:25,392        55258054  316818562580  Execution complete  BACK         2            0     Live

However, for a sizeable proportion of the time, the "Execution complete" status update is received several minutes later, when the market gets resulted. Note that the bet is already fully matched when it reaches the status of "Executable", but it takes almost 9 minutes for the "Exection complete" update to arrive.

Time                       Selection ID  Bet ID        Order status        Side      Matched    Remaining  Trade status
-----------------------  --------------  ------------  ------------------  ------  ---------  -----------  --------------
2023-08-13 10:11:58,988        58973557  None          Pending             BACK            0            2  Live
2023-08-13 10:11:59,296        58973557  316815184967  Executable          BACK            2            0  Pending
2023-08-13 10:20:39,839        58973557  316815184967  Execution complete  BACK            2            0  Live

The issue has been reproduced on other machines and traced down to https://github.com/betcode-org/flumine/blob/master/flumine/order/process.py#L88. The problem is that "order.bet_id is None with status PENDING, current_order.bet_id is set and status is EXECUTION_COMPLETE."

I have just done an experiment with placing FOK bets and they immediately go to EXECUTION_COMPLETE in the order stream, regardless of whether they are matched or not. Here is the data. The first order is matched, the second is cancelled ("sm"/"sc"). They both go straight to "EC" for status, which stands for EXECUTION_COMPLETE.

{'id': '1.216407624',
 'orc': [{'id': 39674645,
          'uo': [{'id': '317109322476',
                  'p': 1.94,
                  's': 1,
                  'side': 'L',
                  'status': 'EC',
                  'pt': 'L',
                  'ot': 'L',
                  'pd': 1692133287000,
                  'md': 1692133287000,
                  'avp': 1.94,
                  'sm': 1,
                  'sr': 0,
                  'sl': 0,
                  'sc': 0,
                  'sv': 0,
                  'rac': '',
                  'rc': 'REG_GGC',
                  'rfo': '',
                  'rfs': ''}],
          'ml': [[1.94, 1]]}]}
{'id': '1.216407624',
 'orc': [{'id': 39674645,
          'uo': [{'id': '317109323812',
                  'p': 1000,
                  's': 1,
                  'side': 'B',
                  'status': 'EC',
                  'pt': 'L',
                  'ot': 'L',
                  'pd': 1692133289000,
                  'sm': 0,
                  'sr': 0,
                  'sl': 0,
                  'sc': 1,
                  'sv': 0,
                  'rac': '',
                  'rc': 'REG_GGC',
                  'rfo': '',
                  'rfs': '',
                  'cd': 1692133289000}]}]}

EXPIRED state appears in the order placement report but not in the order stream. Here are order placement reports for the same test case:

{'status': 'SUCCESS',
 'marketId': '1.216407624',
 'instructionReports': [{'status': 'SUCCESS',
                         'instruction': {'selectionId': 39674645,
                                         'limitOrder': {'size': 1.0,
                                                        'price': 1.94,
                                                        'minFillSize': 0.0,
                                                        'timeInForce': 'FILL_OR_KILL'},
                                         'orderType': 'LIMIT',
                                         'side': 'LAY'},
                         'betId': '317109322476',
                         'placedDate': '2023-08-15T21:01:27.000Z',
                         'averagePriceMatched': 1.94,
                         'sizeMatched': 1.0,
                         'orderStatus': 'EXECUTION_COMPLETE'}]}
{'status': 'SUCCESS',
 'marketId': '1.216407624',
 'instructionReports': [{'status': 'SUCCESS',
                         'instruction': {'selectionId': 39674645,
                                         'limitOrder': {'size': 1.0,
                                                        'price': 1000.0,
                                                        'minFillSize': 0.0,
                                                        'timeInForce': 'FILL_OR_KILL'},
                                         'orderType': 'LIMIT',
                                         'side': 'BACK'},
                         'betId': '317109323812',
                         'placedDate': '2023-08-15T21:01:29.000Z',
                         'averagePriceMatched': 0.0,
                         'sizeMatched': 0.0,
                         'orderStatus': 'EXPIRED'}]}

I do not know what happens to partically matched FOK orders, but I suspect it is one or the other.

In the example in the first post, the order was fully matched, so both the order stream and order placement report should have returned EXECUTION_COMPLETE status. Under no circumstances should this order ever be in EXECUTABLE state.

betcode-org / flumine

Fill or kill orders stuck in executable state #681