Resolve TODO for secondary DB

[pooja1pathak]

Resolve TODO for secondary DB at: https://github.com/orchestracities/ngsi-timeseries-api/blob/af8877cd2b5909c67819d8a93e3b58bce6a61fdb/src/reporter/health.py#L91

[Ravisaketi]

Hi @c0c0n3 @chicco785 i have check for this issue about adding not critical check if a secondary db is configured and i found these comments related this issue https://github.com/orchestracities/ngsi-timeseries-api/issues/377#issuecomment-994316148 and I'm not sure about secondary db, could you please guide me about secondary db.

[c0c0n3]

Hi @Necravisaketi, sorry for getting back to you sooo late on this. But as they say, better late than never :-)

Quantum Leap is designed to work with different DB backends simultaneously. At the moment we only support two DBs, namely Crate and Timescale, but we could add more in the future. For any given API call requiring a DB, QL selects a DB backend depending on the tenant (FIWARE service header) the call is for. The algo that does this tenant-to-DB routing is:

1. Load a tenant-to-DB-ID map M.
2. If M[tenant] = db-id exists, then use the DB identified by db-id.
3. Otherwise, use the DB specified by the QL_DEFAULT_DB env var if set or Crate if the var isn't set. When this is the case, we say we're using the "default backend".

The map M gets loaded from QL's config YAML file (QL_CONFIG env var) if that file exists, otherwise M is the empty map---which results in every call using the "default backend" b/c of (3) above.

The YAML configuration file specifies what backend to use for which tenant as well as the default backend to use for any other tenant not explicitly mentioned in the file. Here's an example YAML configuration:

tenants:
    t1:
        backend: Timescale
    t2:
        backend: Crate
    t3:
        backend: Timescale

default-backend: Crate

With this configuration, any NGSI entity coming in for tenant t1 or t3 gets stored in Timescale whereas tenant t2 gets to use Crate. Any tenant other than t1, t2, or t3 gets the default Crate backend.

[c0c0n3]

Now let's talk about "secondary DB". I think this is a bit of a misnomer, since you could be in a situation where half of your tenants use Crate and the other half Timescale. What's the meaning of "secondary" in this case? So let's just drop that term and talk about configured DBs.

So if you want a sane implementation of that health check, you should first determine the set of configured DBs and then ping each DB in the set. How to figure out that set though? Well, the set is given by taking the values in M plus the value of QL_DEFAULT_DB if set and remove duplicates.