orchestracities / ngsi-timeseries-api

QuantumLeap: a FIWARE Generic Enabler to support the usage of NGSIv2 (and NGSI-LD experimentally) data in time-series databases
https://quantumleap.rtfd.io/
MIT License
38 stars 49 forks source link

Data models with StructuredValues cannot be read from Crate DB #339

Closed ilarimikkonen closed 3 years ago

ilarimikkonen commented 4 years ago

There are some data models out there where there are StructuredValues. This works fine with Orion context Broker, but when the values go with notification to QL/Crate, those values cannot be read (or at least we have not come up with a way) from Crate DB

What is the recommendation how to deal with StructuredValues in FIWARE data models.

Here is one example:

https://github.com/smart-data-models/dataModel.UrbanMobility/blob/a6b3bc03a459a106138489faab1b91ee6942cb9e/PublicTransportStop/README.md

c0c0n3 commented 4 years ago

When a notification contains an entity with an attribute of StructuredValue type, QL saves that attribute's value in the in the corresponding CrateDB entity table with a type of object. This bit should work fine, if you come across an instance where it does not, please file a bug with QL ;-)

You should be able to retrieve the attribute as StructuredValue too, e.g. querying refPublicTransportRoute in the above example through the QL API should return a list of arrays:

...
values: [
    [ "urn:ngsi-ld:PublicTransportRoute:santander:transport:busLine:N3",
      "urn:ngsild:PublicTransportRoute:santander:transport:busLine:N4"],
    ["some:other:id", "and:yet:another"]
]
...

Again, if this doesn't work for you, we're to blame, please open an issue for it ;-)

But you can't query StructuredValue values through the QL API. We don't support that at the moment. So for example if you wanted to know how urn:ngsi-ld:PublicTransportRoute:santander:transport:busLine:N3 varied over time, you'd have to

  1. query CrateDB directly; or
  2. filter the result set returned by QL in your client.

Keep in mind that depending on how much data you're pulling from the DB, option 2 may be awfully inefficient. As usual there's more than one way to skin a cat, but (1) and (2) are probably what most people do? For the rest I'm not aware of any "official" recommendations, but I could be wrong :-)

chicco785 commented 3 years ago

There was no follow-up, so we close this issue.