Apps aren't loaded yet while testing ib

[ben1628]

I'm trying to make sure my IB api is working, so I just do the following in my notebook, and may try to do a simple test on IB.

import numpy as np
import vectorbtpro as vbt

import sys
sys.path.append("..")

from orders import ib

and I get this error

AppRegistryNotReady                       Traceback (most recent call last)
Cell In[6], line 7
      4 import sys
      5 sys.path.append("..")
----> 7 from orders import ib

File ~/py-trading-bot/py-trading-bot/jupyter_scripts/../orders/ib.py:27
     24 logger = logging.getLogger(__name__)
     25 logger_trade = logging.getLogger('trade')
---> 27 from orders.models import (Action, Order, ActionCategory, Excluded, Strategy,
     28                            action_to_etf,
     29                            get_pf, get_order_capital, period_YF_to_ib,
     30                            exchange_to_index_symbol
     31                            )
     32 #Module to handle IB connection
     33 
     34 ##Part of the code that should be in _settings.py in vbt
     35 '''        
     36 #from vectorbtpro.utils.config import ChildDict, Config, FrozenConfig                   
     37 data = ChildDict(
   (...)
     45 )
     46 '''

File ~/py-trading-bot/py-trading-bot/jupyter_scripts/../orders/models.py:126
    123     else:
    124         return symbol #action in this case
--> 126 class Currency(models.Model):
    127     name=models.CharField(max_length=100, blank=False)
    128     symbol=models.CharField(max_length=100, blank=False,default="A")

File ~/.vbt/lib/python3.10/site-packages/django/db/models/base.py:127, in ModelBase.__new__(cls, name, bases, attrs, **kwargs)
    124 app_label = None
    126 # Look for an application configuration to attach the model to.
--> 127 app_config = apps.get_containing_app_config(module)
    129 if getattr(meta, "app_label", None) is None:
    130     if app_config is None:

File ~/.vbt/lib/python3.10/site-packages/django/apps/registry.py:260, in Apps.get_containing_app_config(self, object_name)
    251 def get_containing_app_config(self, object_name):
    252     """
    253     Look for an app config containing a given object.
    254 
   (...)
    258     Return None if the object isn't in any registered app config.
    259     """
--> 260     self.check_apps_ready()
    261     candidates = []
    262     for app_config in self.app_configs.values():

File ~/.vbt/lib/python3.10/site-packages/django/apps/registry.py:138, in Apps.check_apps_ready(self)
    134 # If "not ready" is due to unconfigured settings, accessing
    135 # INSTALLED_APPS raises a more helpful ImproperlyConfigured
    136 # exception.
    137 settings.INSTALLED_APPS
--> 138 raise AppRegistryNotReady("Apps aren't loaded yet.")

AppRegistryNotReady: Apps aren't loaded yet.

any ideas?

I am using vectorbtpro 1.9.0 and merge the IBData you have, at the time, IBData wasn't updated with 1.9.0, I hope it's complete

[psemdel]

There are part of the module that requires Django to run to work. To test for instance, you cannot do unittest ... , you need to do python manage.py test. Django running means that the DB is running and so on. That's also the reason why I have such a distinction between strat, stratL, stratP... the first two don't need Django to run, they are there for backtesting. The "P" works however only in production and need Django to run.

Here in your case, ib import from orders.models, which is a clear Django "thing". Python alone cannot interpret what is in this model.

To do what you want to do. Start Django, (python manage.py runserver), and put the script you want to run in (let's say reporting/)view.py in the test function. You can then trigger it with localhost:8000/test.

[ben1628]

There are two files with the IBData changes:

1/ opt_packages.py

Add ib_insync=dict(name="ib_insync", link="https://github.com/erdewit/ib_insync"), after riskfolio

2/ Custom.py

Add

"IBData" after "TVData"

and then append ( the branch vectorbt.pro-IB_support, with version 1.8.2)

ib_global={"connected":False, "client":None}
class IBData(RemoteData):
...

at the end of the file after

TVData.override_column_config_doc(__pdoc__) I hope that's all the changes in vectrobtpro

[psemdel]

You can look at https://github.com/polakowo/vectorbt.pro/tree/IB_support_2/contrib/IB_support (a bit sad, that it is not merged yet).

You can put it in vectorbt or in the bot, it is exactly the same (and thx again for the push, to put it in vbt!)

[ben1628]

Unfortunately, I don't have access to Vectorbtpro after version 1.90, I don't need those new features. I hope IB_Support you have previously is all the changes you have made.

The changes I made was noted previously in this comment.

[ben1628]

Here is the IBData that I have in data/custom.py

__all__ = [
   "AlpacaData",
    "PolygonData",
    "AVData",
    "NDLData",
    "TVData",
    "IBData"

]

and the class IBData

ib_global={"connected":False, "client":None}

class IBData(RemoteData):
    _setting_keys: tp.SettingsKeys = dict(custom="data.custom.ib")

    @classmethod
    def connect(cls):
        ib_cfg = cls.get_settings(key_id="custom")

        if not cls.client.isConnected():
            clientID=1
            while clientID<=100:
                try:
                    cls.client.connect(host=ib_cfg['localhost'], port=ib_cfg['port'], clientId=clientID)
                    break
                except:                    
                    clientID+=1
                    pass
        if cls.client.isConnected():
            ib_global["connected"]=True
        else:
            warnings.warn("connection to IB failed, check that IB is started")   

    @classmethod
    def resolve_client(cls, client: tp.Optional[tp.Any] = None, **client_config) -> tp.Any:

        from vectorbtpro.utils.opt_packages import assert_can_import
        assert_can_import("ib_insync")
        from ib_insync import IB
        import asyncio

        if client is None and "cls.client" not in locals(): #create a new connection
            if ib_global["client"] is None:
                loop = asyncio.new_event_loop()
                asyncio.set_event_loop(loop)

                cls.client=IB()
                cls.connect()
                ib_global["client"]=cls.client
            else:
                cls.client=ib_global["client"]
        elif client is not None:
            cls.client=client

        return cls.client

    @classmethod 
    def get_contract_ib(cls, symbol,exchange,index):
        #resolve client???

        from ib_insync import Stock, Index
        if index:
            return Index(exchange=exchange,symbol=symbol)
        elif exchange=="NASDAQ":
            return Stock(symbol,"SMART", primaryExchange='NASDAQ')
        else:
            return Stock(symbol,exchange)
        return None

    @classmethod
    def fetch_symbol(
        cls,
        symbol: str, 
        client: tp.Optional[tp.Any] = None,
        client_config: tp.KwargsLike = None,
        period: tp.Optional[str] = None,
        start: tp.Optional[tp.DatetimeLike] = None,
        end: tp.Optional[tp.DatetimeLike] = None,
        timeframe: tp.Optional[str] = None,
        indexes: tp.Optional[dict] = None,
        exchanges: tp.Optional[dict] = None,
        ) -> tp.Any:

        from vectorbtpro.utils.opt_packages import assert_can_import
        assert_can_import("ib_insync")
        from ib_insync import util

        exchange="SMART" #default
        if exchanges is not None:
            if symbol in exchanges:
                exchange=exchanges[symbol]

        index=False
        if indexes is not None:
            if symbol in indexes:
                index=indexes[symbol]        

        if client_config is None:
            client_config = {}
        cls.resolve_client(client=client, **client_config)

        if ib_global["connected"]:
            contract=cls.get_contract_ib(symbol,exchange,index)
            #check period and timeframe
            bars = cls.client.reqHistoricalData(
                    contract,
                    endDateTime='',
                    durationStr=period, #"10 D","1 M"
                    barSizeSetting=timeframe, #"1 day", "1 min"
                    whatToShow='TRADES',
                    useRTH=True,
                    formatDate=1)
            df=util.df(bars)

            if df is not None:
                df.rename(
                    columns={
                        "date":"Date",
                        "open": "Open",
                        "high": "High",
                        "low": "Low",
                        "close": "Close",
                        "volume": "Volume",
                        "average":"Average",
                        "barCount": 'BarCount',
                    },
                    inplace=True,
                )
                df=df.set_index('Date')

            return df

    @classmethod
    def get_last_price(cls,contract):
        timeout=2
        t=0
        cls.resolve_client(client=None)
        m_data = cls.client.reqMktData(contract)
        while m_data.last != m_data.last and t<timeout:  #Wait until data is in. 
            t+=0.01
            cls.client.sleep(0.01)
        if t==timeout:
            m_data.last=0
        cls.client.cancelMktData(contract)
        return m_data.last

[psemdel]

In _settings.py you also need to add in data

    ib=FrozenConfig(
        localhost='127.0.0.1',
        port=7496,
        ),

[ben1628]

Okay, it is now talking to IB, which is great. I tried the half a month report and it's calling IB without problem.

Now, I just have to figure out when it's not calling IB when I click Start_Bot. I ctrl-click AAPL and TSLA on normal strategy. However, telegram is working, but not seeing the program calling IB to get the real time market data.

However, it's a good start, I'm slowly getting there and in the process understanding what's going on, which is essential.

Again, thanks for your help!

P.S. My IB gateway is running on a windows 10, so I have a good GUI there. While trading-bot is on linux