orchestracities / ngsi-timeseries-api

QuantumLeap: a FIWARE Generic Enabler to support the usage of NGSIv2 (and NGSI-LD experimentally) data in time-series databases
https://quantumleap.rtfd.io/
MIT License
38 stars 49 forks source link

Validation failure: Cannot cast to type string #288

Closed glagnar closed 4 years ago

glagnar commented 4 years ago

Describe the bug I am posting this valid NGSI entity and get an error message: Validation failed for category: Cannot cast ['sensor'] to type string]

{
    "id": "urn:ngsi-ld:Device:4D404933-2A6F-456C-8F95-D4F739CC7619",
    "type": "Device",
    "dateObserved": "2020-02-07T21:56:58.000Z",
    "category": ["sensor"],
    "batteryLevel": 1,
    "deviceState": "1:w",
    "controlledProperty": ["location"]
}

I am just following the example here: https://fiware-datamodels.readthedocs.io/en/latest/Device/DeviceModel/doc/spec/index.html

To Reproduce Steps to reproduce the behavior:

  1. Send this payload to '...'
  2. See error

Expected behavior A clear and concise description of what you expected to happen.

Environment (please complete the following information):

Additional context Add any other context about the problem here.

glagnar commented 4 years ago

My greatest problem with this is, that it is a MANDATORY field. It does not pass the schema validation without it.

chicco785 commented 4 years ago

I am not sure what you have currently in the db, can you check the structure? Arrays should be supported, reading the message my guess is that you first stored category as a string and then as an array. Type switching is not supported currently.

glagnar commented 4 years ago

Thanks, you are right - it does seem to be defined as a string, and not string array. Since there is no way to drop colum, or even rename column in CrateDB. What do you suggest I do ? Drop the table ? How do I trigger QuantumLeap to create the table again ?

c0c0n3 commented 4 years ago

Hello :-)

Drop the table

If you don't care about the data you already have in it, sure, drop the table.

create the table again ?

QL will do that for you on seeing the first NGSI entity notification for which there's no corresponding Crate table. Keep in mind that as mentioned earlier, column types are inferred from NGSI attribute types the first time an entity attribute A is received in a notification N. For any subsequent notification for the same entity and containing attribute A, QL expects A to have the same type as in N.

chicco785 commented 4 years ago

you may need also to clean the entry in the related entry in the table "doc"."md_ets_metadata"

c0c0n3 commented 4 years ago

clean the entry in the related entry in the table "doc"."md_ets_metadata"

definitely a good idea. For the record, the old entry should get overwritten with the new entity meta info if the attribute set of the "revised" entity is different than that of the entity stored in the table you want to drop:

https://github.com/smartsdk/ngsi-timeseries-api/blob/dfacb7d84fc533b5e10cb591f74e533089adc0ec/src/translators/crate.py#L359

So after dropping the entity table, you can either:

DELETE FROM md_ets_metadata WHERE table_name = 'your-table'

(safest) or POST an empty entity to the notify endpoint---haven't tested this, so I wouldn't recommend it, but in principle it should work.

Hope this helps!