Seed Price Error Message

[emretezel]

Hi Everyone,

On the latest master branch receiving the following error when seeing prices from IB

RuntimeError: This event loop is already running
ERROR:eventkit.event:Value (137, 162, 'Historical Market Data Service error message:HMDS query returned no data: MBTX2@CME Trades', Contract(secType='FUT', conId=564701848, symbol='MBT', lastTradeDateOrContractMonth='20221125', multiplier='0.1', exchange='CME', currency='USD', localSymbol='MBTX2', tradingClass='MBT')) caused exception for event Event<errorEvent, [[None, <weakref at 0x7f31a7140040; to 'IB' at 0x7f31a717dd60>, <function IB._onError at 0x7f31a7c3fd30>], [None, <weakref at 0x7f31a7189360; to 'ibContractsClient' at 0x7f31a96e0760>, <function ibClient.error_handler at 0x7f31a780d550>], [None, <weakref at 0x7f31a71b8450; to 'ibPriceClient' at 0x7f31a5048f10>, <function ibClient.error_handler at 0x7f31a780d550>]]>
Traceback (most recent call last):
  File "/home/pysystemtrade/opt/pysystemtrade/syscore/cache.py", line 30, in get
    value_from_store = self._get_from_store(key)
  File "/home/pysystemtrade/opt/pysystemtrade/syscore/cache.py", line 50, in _get_from_store
    raise missingData("Missing cache element %s" % key)
syscore.exceptions.missingData: Missing cache element _get_contract_details/(Contract(secType='FUT', conId=564701848, symbol='MBT', lastTradeDateOrContractMonth='20221125', multiplier='0.1', exchange='CME', currency='USD', localSymbol='MBTX2', tradingClass='MBT'),)/{'allow_expired': False}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pysystemtrade/anaconda3/envs/pysystemtrade-prod/lib/python3.8/site-packages/eventkit/event.py", line 189, in emit
    result = func(obj, *args)
  File "/home/pysystemtrade/opt/pysystemtrade/sysbrokers/IB/client/ib_client.py", line 121, in error_handler
    log_to_use = self._get_log_for_contract(contract)
  File "/home/pysystemtrade/opt/pysystemtrade/sysbrokers/IB/client/ib_client.py", line 138, in _get_log_for_contract
    instrument_code = self.get_instrument_code_from_broker_contract_object(
  File "/home/pysystemtrade/opt/pysystemtrade/sysbrokers/IB/client/ib_client.py", line 159, in get_instrument_code_from_broker_contract_object
    broker_identity = self.broker_identity_for_contract(broker_contract_object)
  File "/home/pysystemtrade/opt/pysystemtrade/sysbrokers/IB/client/ib_client.py", line 198, in broker_identity_for_contract
    contract_details = self.get_contract_details(
  File "/home/pysystemtrade/opt/pysystemtrade/sysbrokers/IB/client/ib_client.py", line 217, in get_contract_details
    contract_details = self.cache.get(
  File "/home/pysystemtrade/opt/pysystemtrade/syscore/cache.py", line 32, in get
    value_from_store = self._calculate_and_store(
  File "/home/pysystemtrade/opt/pysystemtrade/syscore/cache.py", line 39, in _calculate_and_store
    value = function_instance(*args, **kwargs)
  File "/home/pysystemtrade/opt/pysystemtrade/sysbrokers/IB/client/ib_client.py", line 237, in _get_contract_details
    return self.ib.reqContractDetails(ib_contract_pattern)
  File "/home/pysystemtrade/anaconda3/envs/pysystemtrade-prod/lib/python3.8/site-packages/ib_insync/ib.py", line 927, in reqContractDetails
    return self._run(self.reqContractDetailsAsync(contract))
  File "/home/pysystemtrade/anaconda3/envs/pysystemtrade-prod/lib/python3.8/site-packages/ib_insync/ib.py", line 308, in _run
    return util.run(*awaitables, timeout=self.RequestTimeout)
  File "/home/pysystemtrade/anaconda3/envs/pysystemtrade-prod/lib/python3.8/site-packages/ib_insync/util.py", line 332, in run
    result = loop.run_until_complete(task)
  File "/home/pysystemtrade/anaconda3/envs/pysystemtrade-prod/lib/python3.8/asyncio/base_events.py", line 592, in run_until_complete
    self._check_running()
  File "/home/pysystemtrade/anaconda3/envs/pysystemtrade-prod/lib/python3.8/asyncio/base_events.py", line 552, in _check_running
    raise RuntimeError('This event loop is already running')

[robcarver17]

Discussed here. Not techically an error.

[tgibson11]

I think I've tracked down the cause of this problem.

1) Request contract details from IB 2) Contract not found 3) Attempt to handle error 4) _get_log_for_contract (ib_client, 138) 5) Try to request contract details again (ib_client, 198)

You used to just grab the IB symbol from the contract object, then reverse lookup to get the instrument symbol

I assume you changed that to support different instruments having the same IB symbol.

You already have an IB contract object, so it seems like you should be able to get what you need from there, then lookup the instrument symbol from config. I'm not sure why you are trying to request contract details from IB (and that seems like a lot of overhead just to log an error).

[robcarver17]

I wasn't sure if the ib contract object would already include the exchange

[robcarver17]

OK if you get the contract object eg from positions, it certainly doesn't include the exchange. You need to reqContractDetails.

[robcarver17]

I've implemented another fix for this, but @tgibson11 won't like it .. it involves return missing_contract rather than raising an exception if the instrument isn't found in the config.

[robcarver17]

Think it's fixed now.

[tgibson11]

Yeah...I don't like this fix.

I would have preferred to make the reverse lookup from config smarter: you don't ALWAYS need all those attributes.

[robcarver17]

Well..... I'm happy to hear suggestions.

[tgibson11]

I may look into it some more today if I have time.

I also have an error backing up historic contract positions that I want to figure out first.

[robcarver17]

I just checked something in that might fix that backup

[tgibson11]

It also seems like this doesn't fix the original problem.

You can verify by running update_sampled_contracts.

The "server" side of the IB client receives an error from the Gateway regardless of how you handle exceptions in pysystemtrade. It's the second attempt to call reqContractDetails that is the problem (the attempt itself - not how you handle the result).

I will definitely look at this more.

Question: the BEST solution IMO, would be avoid all this in the first place by not having to jump through hoops to get a logger. What are your thought on that? Maybe include the ibContract object in the message, but don't worry about setting all the log attributes in this case?

[tgibson11]

I just checked something in that might fix that backup

Thanks. Will check it out.

[robcarver17]

It also seems like this doesn't fix the original problem.

You can verify by running update_sampled_contracts.

The "server" side of the IB client receives an error from the Gateway regardless of how you handle exceptions in pysystemtrade. It's the second attempt to call reqContractDetails that is the problem (the attempt itself - not how you handle the result).

I will definitely look at this more.

Question: the BEST solution IMO, would be avoid all this in the first place by not having to jump through hoops to get a logger. What are your thought on that? Maybe include the ibContract object in the message, but don't worry about setting all the log attributes in this case?

Well the original intention here was to be able to label error messages with the correct instrument code, rather than the IB contract. But it does seems like that is more trouble than it's worth.

[tgibson11]

I just checked something in that might fix that backup

First problem fixed, but now similar problem for strategy positions

[tgibson11]

Correction: the error is still on historic contracts.

Traceback (most recent call last): File "/home/todd/pysystemtrade/sysproduction/linux/scripts/run.py", line 67, in func(*args, **kwargs) File "/home/todd/python3.8_venv/lib/python3.8/site-packages/sysproduction/backup_arctic_to_csv.py", line 58, in backup_arctic_to_csv backup_object.backup_arctic_to_csv() File "/home/todd/python3.8_venv/lib/python3.8/site-packages/sysproduction/backup_arctic_to_csv.py", line 85, in backup_arctic_to_csv backup_contract_position_data(backup_data) File "/home/todd/python3.8_venv/lib/python3.8/site-packages/sysproduction/backup_arctic_to_csv.py", line 357, in backup_contract_position_data data.csv_contract_position.overwrite_position_series_for_contract_object_without_checking( File "/home/todd/python3.8_venv/lib/python3.8/site-packages/sysdata/production/historic_contract_positions.py", line 244, in overwrite_position_series_for_contract_object_without_checking self._write_updated_position_series_for_contract_object( TypeError: _write_updated_position_series_for_contract_object() got an unexpected keyword argument 'updated_series'

[robcarver17]

I've decided to just pull the functionality, so no log is created.

[robcarver17]

@tgibson11 think that's also fixed now.

[tgibson11]

I like how the log message comes out now. It seems like a good tradeoff for the reduced complexity, IMO.

[emretezel]

Have the changes been pushed to master, still getting original error message when for example updating sampled contracts.

[tgibson11]

No, it is only in develop. But develop is in pretty good working order right now. Should be safe to grab if you want to.