search

telefonicaid / iotagent-ul

IoT Agent for a UltraLight 2.0 based protocol (with HTTP, MQTT and AMQP transports)
https://fiware-iotagent-ul.rtfd.io/
GNU Affero General Public License v3.0
38 stars 55 forks source link

Device is created when sending measures to non existing device #216

Closed gobaldia closed 1 year ago

gobaldia commented 7 years ago

We've discovered a bug in the agent. If we try to send a measure to a non existing device, we get a 404 - DEVICE_NOT_FOUND error but at the same time a device without any attribute is created in IoTA's and Orion CB's database. When I say a device without any attribute I refer to the following:

{
    "device_id": "test",
    "service": "MyService",
    "service_path": "/MyServicePath",
    "entity_name": "MyEntity:test",
    "entity_type": "MyEntity",
    "attributes": [],
    "lazy": [],
    "commands": [],
    "static_attributes": []
}

This is a very important bug, because it's really simple to create as many devices as something want and that could eat our database space.

gobaldia commented 7 years ago

By the way, I don't have appendMode enabled.

fgalan commented 7 years ago

Need to be analyzed. On the meanwhile, "bug" label has been added.

fgalan commented 7 years ago

By the way, I don't have appendMode enabled.

I understand that this means appendMode=false or missing appendMode (which implicitely means also false). Is my interpretation correct?

gobaldia commented 7 years ago

You're totally right @fgalan.

AlvaroVega commented 6 years ago

Could you provide please a full test about this issue? Autoprovisioning is allowed if you are using for example a proper apikey.

KeshavSoni2511 commented 1 year ago

Hi @gobaldia, Please share service key details that you have created while sending measurements to non existing device. I have send measurements to non existing device but an existing service key the device was auto provisioned but in case of non existing service key the device is not created and throw an error of msg=Error [DEVICE_GROUP_NOT_FOUND].

Hi @AlvaroVega , What's your opinion on this issue? Thanks

AlvaroVega commented 1 year ago

In my opinion this is not a bug, autoprovision allow send measures of a non provisoned device and the nprovisioned is created. You only just need a valid apikey (from a configurationGroup already provisioned)

KeshavSoni2511 commented 1 year ago

Hi, @AlvaroVega , I think we can close this issue as nothing is left to do in this issue and also I have shared my findings.

fgalan commented 1 year ago

@gobaldia as original author of the issue, what do you think about closing it?

KeshavSoni2511 commented 1 year ago

Hi @fgalan , I think @gobaldia is not active since 2017.

mapedraza commented 1 year ago

As @AlvaroVega noted, the fact of creating devices (autoprovision) is not a bug, since you need to know the APIKEY. If you don't trust your device provider, you can always disable autoprovision.

What is a bit confusing is the given error: 404 - DEVICE_NOT_FOUND. It makes sense the IoTA give you that error only if autoprovision = false. Otherwise, no error message should be returned.

@Keshav-NEC could you please check which response the IoTA gives when sending data from unprovisioned devices? (consider also appendMode env var. I suggest you to have a look to this doc section: https://github.com/telefonicaid/iotagent-node-lib/blob/master/doc/api.md#measurement-persistence-options

AlvaroVega commented 1 year ago

In order to reproduce this bugit is needed to provide iotagent configuration, group/device provision and measure sent.

KeshavSoni2511 commented 1 year ago

Hi @mapedraza , when I send measurement to non existing device while no service api key is created, the error it gives msg=Error [DEVICE_GROUP_NOT_FOUND] & device is not created. When I send measurements to non existing device with existing service key the error iotagent through msg=Error [ENTITY_GENERIC_ERROR] & device is created but without measurements.

And I have one query , are you talking about append mode in lib/commonConfig.js? Please let me know.

mapedraza commented 1 year ago

Hi @mapedraza , when I send measurement to non existing device while no service api key is created, the error it gives msg=Error [DEVICE_GROUP_NOT_FOUND] & device is not created. When I send measurements to non existing device with existing service key the error iotagent through msg=Error [ENTITY_GENERIC_ERROR] & device is created but without measurements.

As @AlvaroVega mentioned above, in order to reproduce it, you should share:

And I have one query , are you talking about append mode in lib/commonConfig.js? Please let me know.

append mode you can check here (config.js or env vars)

KeshavSoni2511 commented 1 year ago

Hi @mapedraza as requested by you Config group provision

curl -iX POST \
  'http://localhost:4041/iot/services' \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "services": [
   {
     "apikey":      "4jggokgpepnvsb2uv4s40d59ov",
     "cbroker":     "http://orion:1026",
     "entity_type": "Thing",
     "resource":    "/iot/d"
   }
 ]
}'

And I have not done any changes for that in config.js of both iotagent nodelib & iotagent ul

As, I can find auto provision true in devices/deviceService.js , So, I can change it to false to check the auto provision. Need your opinion on this.

fgalan commented 1 year ago

Answer in above comment is ok with regards to

Config group provision

However, in order to reproduced the issue with full precission we also need this

IoT agent config.js or environment variables configurations

as @mapedraza already has told.

KeshavSoni2511 commented 1 year ago

Hi @fgalan @mapedraza, please find

IoT agent config.js

I have changed port to 4041

config.txt

fgalan commented 1 year ago

@Keshav-NEC could you also provide the request you use to send the UL measure to the IOTA-UL, please? (in curl format if possible). Thanks!

fgalan commented 1 year ago

@Keshav-NEC anyway take into account that we are now in the middle of a refactor work which could change the way in which devices are provisioned. Take it into account, please.

KeshavSoni2511 commented 1 year ago

Hi @fgalan, please find the request I use to send the UL measure to IOTA-UL

curl -iX POST \
  'http://localhost:7896/iot/d?k=4jggokgpepnvsb2uv4s40d59ov&i=motion001' \
  -H 'Content-Type: text/plain' \
  -d 'c|1'
AlvaroVega commented 1 year ago

HI, sent measure https://github.com/telefonicaid/iotagent-ul/issues/216#issuecomment-1589746977 with group https://github.com/telefonicaid/iotagent-ul/issues/216#issuecomment-1578485095 I've obtained the following logs (wihtout any error, as exptected)

BTW, my config is using IOTA_APPEND_MODE=ftrue

time=2023-06-14T07:59:19.475Z | lvl=DEBUG | corr=e19aaf79-c298-4aad-9206-05f0d1ff7917 | trans=e19aaf79-c298-4aad-9206-05f0d1ff7917 | op=IOTAUL.HTTP.Binding | from=n/a | srv=n/a | subsrv=n/a | msg=Parsing payload [c|1] | comp=IoTAgent time=2023-06-14T07:59:19.476Z | lvl=DEBUG | corr=e19aaf79-c298-4aad-9206-05f0d1ff7917 | trans=e19aaf79-c298-4aad-9206-05f0d1ff7917 | op=IOTAUL.HTTP.Binding | from=n/a | srv=n/a | subsrv=n/a | msg=Parsed data: [[{"c":"1"}]] | comp=IoTAgent time=2023-06-14T07:59:19.476Z | lvl=DEBUG | corr=e19aaf79-c298-4aad-9206-05f0d1ff7917 | trans=e19aaf79-c298-4aad-9206-05f0d1ff7917 | op=IOTAUL.HTTP.Binding | from=n/a | srv=n/a | subsrv=n/a | msg=Processing multiple HTTP measures for device [motion001] with apiKey ["4jggokgpepnvsb2uv4s40d59ov"] | comp=IoTAgent time=2023-06-14T07:59:19.476Z | lvl=DEBUG | corr=e19aaf79-c298-4aad-9206-05f0d1ff7917 | trans=e19aaf79-c298-4aad-9206-05f0d1ff7917 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Looking for group params ["resource","apikey"] with queryObj {"resource":"/iot/d","apikey":"4jggokgpepnvsb2uv4s40d59ov"} | comp=IoTAgent time=2023-06-14T07:59:19.478Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Device group data found: {"_id":"6489723171b81e4250012c14","commands":[],"staticAttributes":[],"attributes":[],"resource":"/iot/d","apikey":"4jggokgpepnvsb2uv4s40d59ov","type":"Thing","service":"smartcity","subservice":"/","description":"miUL"} | comp=IoTAgent time=2023-06-14T07:59:19.478Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.MongoDBDeviceRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Looking for device with id [motion001]. | comp=IoTAgent time=2023-06-14T07:59:19.480Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.MongoDBDeviceRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Device [motion001] not found. | comp=IoTAgent time=2023-06-14T07:59:19.480Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.MongoDBDeviceRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Looking for device with id [motion001]. | comp=IoTAgent time=2023-06-14T07:59:19.481Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.MongoDBDeviceRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Device [motion001] not found. | comp=IoTAgent time=2023-06-14T07:59:19.481Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Looking for group params ["service","subservice","type","apikey"] with queryObj {"service":"smartcity","subservice":"/","type":"Thing","apikey":"4jggokgpepnvsb2uv4s40d59ov"} | comp=IoTAgent time=2023-06-14T07:59:19.483Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Device group data found: {"_id":"6489723171b81e4250012c14","commands":[],"staticAttributes":[],"attributes":[],"resource":"/iot/d","apikey":"4jggokgpepnvsb2uv4s40d59ov","type":"Thing","service":"smartcity","subservice":"/","description":"miUL"} | comp=IoTAgent time=2023-06-14T07:59:19.483Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.DeviceService | from=n/a | srv=smartcity | subsrv=/ | msg=Device name not found, falling back to deviceType:deviceId [undefined] | comp=IoTAgent time=2023-06-14T07:59:19.484Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.DeviceService | from=n/a | srv=smartcity | subsrv=/ | msg=deviceData before merge with conf: {"id":"motion001","service":"smartcity","subservice":"/","type":"Thing","protocol":"IoTA-UL","transport":"HTTP","apikey":"4jggokgpepnvsb2uv4s40d59ov","name":"Thing:motion001"} defaults: [null,null,[],[],[],[],[]] fields: ["lazy","active","staticAttributes","commands","subscriptions"] configuration {"_id":"6489723171b81e4250012c14","commands":[],"staticAttributes":[],"attributes":[],"resource":"/iot/d","apikey":"4jggokgpepnvsb2uv4s40d59ov","type":"Thing","service":"smartcity","subservice":"/","description":"miUL"} | comp=IoTAgent time=2023-06-14T07:59:19.484Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.DeviceService | from=n/a | srv=smartcity | subsrv=/ | msg=deviceData after merge with conf: {"id":"motion001","service":"smartcity","subservice":"/","type":"Thing","protocol":"IoTA-UL","transport":"HTTP","apikey":"4jggokgpepnvsb2uv4s40d59ov","name":"Thing:motion001","lazy":null,"active":[],"staticAttributes":[],"commands":[],"subscriptions":[]} | comp=IoTAgent time=2023-06-14T07:59:19.484Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.DeviceService | from=n/a | srv=smartcity | subsrv=/ | msg=Registering device into NGSI Service: { "id": "motion001", "service": "smartcity", "subservice": "/", "type": "Thing", "protocol": "IoTA-UL", "transport": "HTTP", "apikey": "4jggokgpepnvsb2uv4s40d59ov", "name": "Thing:motion001", "lazy": null, "active": [], "staticAttributes": [], "commands": [], "subscriptions": [] } | comp=IoTAgent time=2023-06-14T07:59:19.484Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Registration | from=n/a | srv=smartcity | subsrv=/ | msg=Registration with Context Provider is not needed. Device without lazy atts or commands | comp=IoTAgent time=2023-06-14T07:59:19.484Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Devices-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=deviceData: {"id":"motion001","service":"smartcity","subservice":"/","type":"Thing","protocol":"IoTA-UL","transport":"HTTP","apikey":"4jggokgpepnvsb2uv4s40d59ov","name":"Thing:motion001","lazy":null,"active":[],"staticAttributes":[],"commands":[],"subscriptions":[]} | comp=IoTAgent time=2023-06-14T07:59:19.484Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Devices-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=config.timestamp undefined true | comp=IoTAgent time=2023-06-14T07:59:19.485Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Devices-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=Creating initial entity in the Context Broker: { "url": "http://iot-orion:1026/v2/entities?options=upsert", "method": "POST", "json": { "id": "Thing:motion001", "type": "Thing", "TimeInstant": { "type": "DateTime", "value": "2023-06-14T07:59:19.485Z" } }, "headers": { "fiware-service": "smartcity", "fiware-servicepath": "/", "fiware-correlator": "000000a7-a388-4f19-a4f3-7140c1adf3b0" } } | comp=IoTAgent time=2023-06-14T07:59:19.485Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.RestUtils | from=n/a | srv=smartcity | subsrv=/ | msg=executeWithSecurity | comp=IoTAgent time=2023-06-14T07:59:19.486Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Looking for group params ["type"] with queryObj {"type":"Thing"} | comp=IoTAgent time=2023-06-14T07:59:19.487Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Device group data found: {"_id":"6489723171b81e4250012c14","commands":[],"staticAttributes":[],"attributes":[],"resource":"/iot/d","apikey":"4jggokgpepnvsb2uv4s40d59ov","type":"Thing","service":"smartcity","subservice":"/","description":"miUL"} | comp=IoTAgent time=2023-06-14T07:59:19.487Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Request | from=n/a | srv=smartcity | subsrv=/ | msg=Options: { "url": "http://iot-orion:1026/v2/entities?options=upsert", "method": "POST", "json": { "id": "Thing:motion001", "type": "Thing", "TimeInstant": { "type": "DateTime", "value": "2023-06-14T07:59:19.485Z" } }, "headers": { "fiware-service": "smartcity", "fiware-servicepath": "/", "fiware-correlator": "000000a7-a388-4f19-a4f3-7140c1adf3b0" } } | comp=IoTAgent time=2023-06-14T07:59:19.493Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Request | from=n/a | srv=smartcity | subsrv=/ | msg=Response "" | comp=IoTAgent time=2023-06-14T07:59:19.493Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Devices-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=Initial entity created successfully. | comp=IoTAgent time=2023-06-14T07:59:19.496Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.MongoDBDeviceRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Storing device with id [motion001] and type [Thing] | comp=IoTAgent time=2023-06-14T07:59:19.500Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IOTAUL.IoTUtils | from=n/a | srv=smartcity | subsrv=/ | msg=deviceData before merge with conf: {"lazy":null,"commands":[],"staticAttributes":[],"subscriptions":[],"_id":"64897357c3bdb8542f07cd88","creationDate":"2023-06-14T07:59:19.493Z","id":"motion001","type":"Thing","name":"Thing:motion001","service":"smartcity","subservice":"/","apikey":"4jggokgpepnvsb2uv4s40d59ov","protocol":"IoTA-UL","transport":"HTTP","v":0} | comp=IoTAgent time=2023-06-14T07:59:19.501Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IOTAUL.IoTUtils | from=n/a | srv=smartcity | subsrv=/ | msg=deviceData after merge with conf: {"lazy":null,"commands":[],"staticAttributes":[],"subscriptions":[],"_id":"64897357c3bdb8542f07cd88","creationDate":"2023-06-14T07:59:19.493Z","id":"motion001","type":"Thing","name":"Thing:motion001","service":"smartcity","subservice":"/","apikey":"4jggokgpepnvsb2uv4s40d59ov","protocol":"IoTA-UL","transport":"HTTP","__v":0,"internalAttributes":null,"active":[]} | comp=IoTAgent time=2023-06-14T07:59:19.501Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.NGSIService | from=n/a | srv=smartcity | subsrv=/ | msg=executeWithDeviceInfo entityName Thing:motion001 type Thing apikey attributes [{"name":"c","type":"string","value":"1"}] deviceInformation {"lazy":null,"commands":[],"staticAttributes":[],"subscriptions":[],"_id":"64897357c3bdb8542f07cd88","creationDate":"2023-06-14T07:59:19.493Z","id":"motion001","type":"Thing","name":"Thing:motion001","service":"smartcity","subservice":"/","apikey":"4jggokgpepnvsb2uv4s40d59ov","protocol":"IoTA-UL","transport":"HTTP","v":0,"internalAttributes":null,"active":[]} | comp=IoTAgent time=2023-06-14T07:59:19.501Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Looking for group params ["type"] with queryObj {"type":"Thing"} | comp=IoTAgent time=2023-06-14T07:59:19.502Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Device group data found: {"_id":"6489723171b81e4250012c14","commands":[],"staticAttributes":[],"attributes":[],"resource":"/iot/d","apikey":"4jggokgpepnvsb2uv4s40d59ov","type":"Thing","service":"smartcity","subservice":"/","description":"miUL"} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 called with: entityName=Thing:motion001 attributes=[{"name":"c","type":"string","value":"1"}] typeInformation={"lazy":null,"commands":[],"staticAttributes":[],"subscriptions":[],"_id":"64897357c3bdb8542f07cd88","creationDate":"2023-06-14T07:59:19.493Z","id":"motion001","type":"Thing","name":"Thing:motion001","service":"smartcity","subservice":"/","apikey":"4jggokgpepnvsb2uv4s40d59ov","protocol":"IoTA-UL","transport":"HTTP","v":0,"internalAttributes":null,"active":[]} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.NGSIUtils | from=n/a | srv=smartcity | subsrv=/ | msg=typeInformation {"lazy":null,"commands":[],"staticAttributes":[],"subscriptions":[],"_id":"64897357c3bdb8542f07cd88","creationDate":"2023-06-14T07:59:19.493Z","id":"motion001","type":"Thing","name":"Thing:motion001","service":"smartcity","subservice":"/","apikey":"4jggokgpepnvsb2uv4s40d59ov","protocol":"IoTA-UL","transport":"HTTP","__v":0,"internalAttributes":null,"active":[]} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 idTypeSSS are [{"name":"id","type":"String","value":"motion001"},{"name":"type","type":"String","value":"Thing"},{"name":"service","type":"String","value":"smartcity"},{"name":"subservice","type":"String","value":"/"},{"name":"entity_name","type":"String","value":"Thing:motion001"}] | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 pre-initial non-explicitAttrs payload={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}}],"actionType":"append"} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 initial payload={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}}],"actionType":"append"} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 initial ctxt {"c":1,"id":"motion001","type":"Thing","service":"smartcity","subservice":"/","entity_name":"Thing:motion001"} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 currentEntity {"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 currentEntity sorted {"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 procesing j="id" attr=undefined ctxt={"c":1,"id":"motion001","type":"Thing","service":"smartcity","subservice":"/","entity_name":"Thing:motion001"} newAttr="Thing:motion001" | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 after procesing attr="id" current entity={"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}} current payload={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}}],"actionType":"append"} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 procesing j="type" attr=undefined ctxt={"c":1,"id":"motion001","type":"Thing","service":"smartcity","subservice":"/","entity_name":"Thing:motion001"} newAttr="Thing" | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 after procesing attr="type" current entity={"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}} current payload={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}}],"actionType":"append"} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 procesing j="c" attr=undefined ctxt={"c":1,"id":"motion001","type":"Thing","service":"smartcity","subservice":"/","entity_name":"Thing:motion001"} newAttr={"value":"1","type":"string"} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 after procesing attr="c" current entity={"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}} current payload={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}}],"actionType":"append"} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 ending payload={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string","metadata":{"TimeInstant":{"type":"DateTime","value":"2023-06-14T07:59:19.503Z"}}},"TimeInstant":{"type":"DateTime","value":"2023-06-14T07:59:19.503Z"}}],"actionType":"append"} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 payload with native types and without object_id={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":1,"type":"string","metadata":{"TimeInstant":{"type":"DateTime","value":"2023-06-14T07:59:19.503Z"}}},"TimeInstant":{"type":"DateTime","value":"2023-06-14T07:59:19.503Z"}}],"actionType":"append"} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.NGSIUtils | from=n/a | srv=smartcity | subsrv=/ | msg=typeInformation {"lazy":null,"commands":[],"staticAttributes":[],"subscriptions":[],"_id":"64897357c3bdb8542f07cd88","creationDate":"2023-06-14T07:59:19.493Z","id":"motion001","type":"Thing","name":"Thing:motion001","service":"smartcity","subservice":"/","apikey":"4jggokgpepnvsb2uv4s40d59ov","protocol":"IoTA-UL","transport":"HTTP","v":0,"internalAttributes":null,"active":[]} | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=Updating device value in the Context Broker at [http://iot-orion:1026/v2/entities/Thing:motion001/attrs?type=Thing] | comp=IoTAgent time=2023-06-14T07:59:19.503Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=Using the following NGSI v2 request:

{ "url": "http://iot-orion:1026/v2/entities/Thing:motion001/attrs?type=Thing", "method": "POST", "headers": { "fiware-service": "smartcity", "fiware-servicepath": "/" }, "json": { "c": { "value": 1, "type": "string", "metadata": { "TimeInstant": { "type": "DateTime", "value": "2023-06-14T07:59:19.503Z" } } }, "TimeInstant": { "type": "DateTime", "value": "2023-06-14T07:59:19.503Z" } } }

| comp=IoTAgent time=2023-06-14T07:59:19.504Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Request | from=n/a | srv=smartcity | subsrv=/ | msg=Options: { "url": "http://iot-orion:1026/v2/entities/Thing:motion001/attrs?type=Thing", "method": "POST", "headers": { "fiware-service": "smartcity", "fiware-servicepath": "/" }, "json": { "c": { "value": 1, "type": "string", "metadata": { "TimeInstant": { "type": "DateTime", "value": "2023-06-14T07:59:19.503Z" } } }, "TimeInstant": { "type": "DateTime", "value": "2023-06-14T07:59:19.503Z" } } } | comp=IoTAgent time=2023-06-14T07:59:19.507Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Request | from=n/a | srv=smartcity | subsrv=/ | msg=Response "" | comp=IoTAgent time=2023-06-14T07:59:19.507Z | lvl=INFO | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=Received the following response from the CB: Value updated successfully | comp=IoTAgent time=2023-06-14T07:59:19.507Z | lvl=INFO | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IOTAUL.HTTP.Binding | from=n/a | srv=smartcity | subsrv=/ | msg=Multiple measures for device [motion001] with apiKey [4jggokgpepnvsb2uv4s40d59ov] successfully updated | comp=IoTAgent time=2023-06-14T07:59:19.507Z | lvl=DEBUG | corr=000000a7-a388-4f19-a4f3-7140c1adf3b0 | trans=000000a7-a388-4f19-a4f3-7140c1adf3b0 | op=IoTAgentNGSI.DomainControl | from=n/a | srv=smartcity | subsrv=/ | msg=response-time: 134900 | comp=IoTAgent

AlvaroVega commented 1 year ago

But testing the same with IOTA_APPEND_MODE=false I've obtained

msg=MEASURES-002: Couldn't send the updated values to the Context Broker due to an error: {"name":"ENTITY_GENERIC_ERROR","message":"Error accesing entity data for device: Thing:motion001 of type: Thing","details":{"error":"Unprocessable","description":"one or more of the attributes in the request do not exist: [ c ]"},"code":422}

because in that case iotagent uses PATCH to update:

This is the full log:

time=2023-06-14T08:04:44.964Z | lvl=DEBUG | corr=c477acf3-6d69-4fa7-a274-3ee6fa1ff561 | trans=c477acf3-6d69-4fa7-a274-3ee6fa1ff561 | op=IoTAgentNGSI.DomainControl | from=n/a | srv=n/a | subsrv=n/a | msg=response-time: 0 | comp=IoTAgent time=2023-06-14T08:04:52.182Z | lvl=DEBUG | corr=7a55d6f4-8a67-4d69-9c69-3e53ed43ac23 | trans=7a55d6f4-8a67-4d69-9c69-3e53ed43ac23 | op=IOTAUL.HTTP.Binding | from=n/a | srv=n/a | subsrv=n/a | msg=Parsing payload [c|1] | comp=IoTAgent time=2023-06-14T08:04:52.182Z | lvl=DEBUG | corr=7a55d6f4-8a67-4d69-9c69-3e53ed43ac23 | trans=7a55d6f4-8a67-4d69-9c69-3e53ed43ac23 | op=IOTAUL.HTTP.Binding | from=n/a | srv=n/a | subsrv=n/a | msg=Parsed data: [[{"c":"1"}]] | comp=IoTAgent time=2023-06-14T08:04:52.182Z | lvl=DEBUG | corr=7a55d6f4-8a67-4d69-9c69-3e53ed43ac23 | trans=7a55d6f4-8a67-4d69-9c69-3e53ed43ac23 | op=IOTAUL.HTTP.Binding | from=n/a | srv=n/a | subsrv=n/a | msg=Processing multiple HTTP measures for device [motion001] with apiKey ["4jggokgpepnvsb2uv4s40d59ov"] | comp=IoTAgent time=2023-06-14T08:04:52.182Z | lvl=DEBUG | corr=7a55d6f4-8a67-4d69-9c69-3e53ed43ac23 | trans=7a55d6f4-8a67-4d69-9c69-3e53ed43ac23 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Looking for group params ["resource","apikey"] with queryObj {"resource":"/iot/d","apikey":"4jggokgpepnvsb2uv4s40d59ov"} | comp=IoTAgent time=2023-06-14T08:04:52.184Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Device group data found: {"_id":"6489723171b81e4250012c14","commands":[],"staticAttributes":[],"attributes":[],"resource":"/iot/d","apikey":"4jggokgpepnvsb2uv4s40d59ov","type":"Thing","service":"smartcity","subservice":"/","description":"miUL"} | comp=IoTAgent time=2023-06-14T08:04:52.184Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.MongoDBDeviceRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Looking for device with id [motion001]. | comp=IoTAgent time=2023-06-14T08:04:52.185Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.MongoDBDeviceRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Device [motion001] not found. | comp=IoTAgent time=2023-06-14T08:04:52.185Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.MongoDBDeviceRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Looking for device with id [motion001]. | comp=IoTAgent time=2023-06-14T08:04:52.186Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.MongoDBDeviceRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Device [motion001] not found. | comp=IoTAgent time=2023-06-14T08:04:52.186Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Looking for group params ["service","subservice","type","apikey"] with queryObj {"service":"smartcity","subservice":"/","type":"Thing","apikey":"4jggokgpepnvsb2uv4s40d59ov"} | comp=IoTAgent time=2023-06-14T08:04:52.187Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Device group data found: {"_id":"6489723171b81e4250012c14","commands":[],"staticAttributes":[],"attributes":[],"resource":"/iot/d","apikey":"4jggokgpepnvsb2uv4s40d59ov","type":"Thing","service":"smartcity","subservice":"/","description":"miUL"} | comp=IoTAgent time=2023-06-14T08:04:52.187Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.DeviceService | from=n/a | srv=smartcity | subsrv=/ | msg=Device name not found, falling back to deviceType:deviceId [undefined] | comp=IoTAgent time=2023-06-14T08:04:52.187Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.DeviceService | from=n/a | srv=smartcity | subsrv=/ | msg=deviceData before merge with conf: {"id":"motion001","service":"smartcity","subservice":"/","type":"Thing","protocol":"IoTA-UL","transport":"HTTP","apikey":"4jggokgpepnvsb2uv4s40d59ov","name":"Thing:motion001"} defaults: [null,null,[],[],[],[],[]] fields: ["lazy","active","staticAttributes","commands","subscriptions"] configuration {"_id":"6489723171b81e4250012c14","commands":[],"staticAttributes":[],"attributes":[],"resource":"/iot/d","apikey":"4jggokgpepnvsb2uv4s40d59ov","type":"Thing","service":"smartcity","subservice":"/","description":"miUL"} | comp=IoTAgent time=2023-06-14T08:04:52.187Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.DeviceService | from=n/a | srv=smartcity | subsrv=/ | msg=deviceData after merge with conf: {"id":"motion001","service":"smartcity","subservice":"/","type":"Thing","protocol":"IoTA-UL","transport":"HTTP","apikey":"4jggokgpepnvsb2uv4s40d59ov","name":"Thing:motion001","lazy":null,"active":[],"staticAttributes":[],"commands":[],"subscriptions":[]} | comp=IoTAgent time=2023-06-14T08:04:52.187Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.DeviceService | from=n/a | srv=smartcity | subsrv=/ | msg=Registering device into NGSI Service: { "id": "motion001", "service": "smartcity", "subservice": "/", "type": "Thing", "protocol": "IoTA-UL", "transport": "HTTP", "apikey": "4jggokgpepnvsb2uv4s40d59ov", "name": "Thing:motion001", "lazy": null, "active": [], "staticAttributes": [], "commands": [], "subscriptions": [] } | comp=IoTAgent time=2023-06-14T08:04:52.188Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Registration | from=n/a | srv=smartcity | subsrv=/ | msg=Registration with Context Provider is not needed. Device without lazy atts or commands | comp=IoTAgent time=2023-06-14T08:04:52.188Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Devices-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=deviceData: {"id":"motion001","service":"smartcity","subservice":"/","type":"Thing","protocol":"IoTA-UL","transport":"HTTP","apikey":"4jggokgpepnvsb2uv4s40d59ov","name":"Thing:motion001","lazy":null,"active":[],"staticAttributes":[],"commands":[],"subscriptions":[]} | comp=IoTAgent time=2023-06-14T08:04:52.188Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Devices-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=config.timestamp undefined true | comp=IoTAgent time=2023-06-14T08:04:52.188Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Devices-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=Creating initial entity in the Context Broker: { "url": "http://iot-orion:1026/v2/entities?options=upsert", "method": "POST", "json": { "id": "Thing:motion001", "type": "Thing", "TimeInstant": { "type": "DateTime", "value": "2023-06-14T08:04:52.188Z" } }, "headers": { "fiware-service": "smartcity", "fiware-servicepath": "/", "fiware-correlator": "fced1dbc-df68-45c6-9fac-e22f8cecfd8d" } } | comp=IoTAgent time=2023-06-14T08:04:52.188Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.RestUtils | from=n/a | srv=smartcity | subsrv=/ | msg=executeWithSecurity | comp=IoTAgent time=2023-06-14T08:04:52.188Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Looking for group params ["type"] with queryObj {"type":"Thing"} | comp=IoTAgent time=2023-06-14T08:04:52.189Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Device group data found: {"_id":"6489723171b81e4250012c14","commands":[],"staticAttributes":[],"attributes":[],"resource":"/iot/d","apikey":"4jggokgpepnvsb2uv4s40d59ov","type":"Thing","service":"smartcity","subservice":"/","description":"miUL"} | comp=IoTAgent time=2023-06-14T08:04:52.189Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Request | from=n/a | srv=smartcity | subsrv=/ | msg=Options: { "url": "http://iot-orion:1026/v2/entities?options=upsert", "method": "POST", "json": { "id": "Thing:motion001", "type": "Thing", "TimeInstant": { "type": "DateTime", "value": "2023-06-14T08:04:52.188Z" } }, "headers": { "fiware-service": "smartcity", "fiware-servicepath": "/", "fiware-correlator": "fced1dbc-df68-45c6-9fac-e22f8cecfd8d" } } | comp=IoTAgent time=2023-06-14T08:04:52.193Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Request | from=n/a | srv=smartcity | subsrv=/ | msg=Response "" | comp=IoTAgent time=2023-06-14T08:04:52.193Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Devices-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=Initial entity created successfully. | comp=IoTAgent time=2023-06-14T08:04:52.195Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.MongoDBDeviceRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Storing device with id [motion001] and type [Thing] | comp=IoTAgent time=2023-06-14T08:04:52.200Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IOTAUL.IoTUtils | from=n/a | srv=smartcity | subsrv=/ | msg=deviceData before merge with conf: {"lazy":null,"commands":[],"staticAttributes":[],"subscriptions":[],"_id":"648974a446e38536e98f93aa","creationDate":"2023-06-14T08:04:52.193Z","id":"motion001","type":"Thing","name":"Thing:motion001","service":"smartcity","subservice":"/","apikey":"4jggokgpepnvsb2uv4s40d59ov","protocol":"IoTA-UL","transport":"HTTP","v":0} | comp=IoTAgent time=2023-06-14T08:04:52.200Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IOTAUL.IoTUtils | from=n/a | srv=smartcity | subsrv=/ | msg=deviceData after merge with conf: {"lazy":null,"commands":[],"staticAttributes":[],"subscriptions":[],"_id":"648974a446e38536e98f93aa","creationDate":"2023-06-14T08:04:52.193Z","id":"motion001","type":"Thing","name":"Thing:motion001","service":"smartcity","subservice":"/","apikey":"4jggokgpepnvsb2uv4s40d59ov","protocol":"IoTA-UL","transport":"HTTP","__v":0,"internalAttributes":null,"active":[]} | comp=IoTAgent time=2023-06-14T08:04:52.200Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.NGSIService | from=n/a | srv=smartcity | subsrv=/ | msg=executeWithDeviceInfo entityName Thing:motion001 type Thing apikey attributes [{"name":"c","type":"string","value":"1"}] deviceInformation {"lazy":null,"commands":[],"staticAttributes":[],"subscriptions":[],"_id":"648974a446e38536e98f93aa","creationDate":"2023-06-14T08:04:52.193Z","id":"motion001","type":"Thing","name":"Thing:motion001","service":"smartcity","subservice":"/","apikey":"4jggokgpepnvsb2uv4s40d59ov","protocol":"IoTA-UL","transport":"HTTP","v":0,"internalAttributes":null,"active":[]} | comp=IoTAgent time=2023-06-14T08:04:52.200Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Looking for group params ["type"] with queryObj {"type":"Thing"} | comp=IoTAgent time=2023-06-14T08:04:52.202Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=smartcity | subsrv=/ | msg=Device group data found: {"_id":"6489723171b81e4250012c14","commands":[],"staticAttributes":[],"attributes":[],"resource":"/iot/d","apikey":"4jggokgpepnvsb2uv4s40d59ov","type":"Thing","service":"smartcity","subservice":"/","description":"miUL"} | comp=IoTAgent time=2023-06-14T08:04:52.202Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 called with: entityName=Thing:motion001 attributes=[{"name":"c","type":"string","value":"1"}] typeInformation={"lazy":null,"commands":[],"staticAttributes":[],"subscriptions":[],"_id":"648974a446e38536e98f93aa","creationDate":"2023-06-14T08:04:52.193Z","id":"motion001","type":"Thing","name":"Thing:motion001","service":"smartcity","subservice":"/","apikey":"4jggokgpepnvsb2uv4s40d59ov","protocol":"IoTA-UL","transport":"HTTP","v":0,"internalAttributes":null,"active":[]} | comp=IoTAgent time=2023-06-14T08:04:52.202Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.NGSIUtils | from=n/a | srv=smartcity | subsrv=/ | msg=typeInformation {"lazy":null,"commands":[],"staticAttributes":[],"subscriptions":[],"_id":"648974a446e38536e98f93aa","creationDate":"2023-06-14T08:04:52.193Z","id":"motion001","type":"Thing","name":"Thing:motion001","service":"smartcity","subservice":"/","apikey":"4jggokgpepnvsb2uv4s40d59ov","protocol":"IoTA-UL","transport":"HTTP","__v":0,"internalAttributes":null,"active":[]} | comp=IoTAgent time=2023-06-14T08:04:52.202Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 idTypeSSS are [{"name":"id","type":"String","value":"motion001"},{"name":"type","type":"String","value":"Thing"},{"name":"service","type":"String","value":"smartcity"},{"name":"subservice","type":"String","value":"/"},{"name":"entity_name","type":"String","value":"Thing:motion001"}] | comp=IoTAgent time=2023-06-14T08:04:52.202Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 pre-initial non-explicitAttrs payload={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}}],"actionType":"update"} | comp=IoTAgent time=2023-06-14T08:04:52.202Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 initial payload={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}}],"actionType":"update"} | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 initial ctxt {"c":1,"id":"motion001","type":"Thing","service":"smartcity","subservice":"/","entity_name":"Thing:motion001"} | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 currentEntity {"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}} | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 currentEntity sorted {"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}} | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 procesing j="id" attr=undefined ctxt={"c":1,"id":"motion001","type":"Thing","service":"smartcity","subservice":"/","entity_name":"Thing:motion001"} newAttr="Thing:motion001" | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 after procesing attr="id" current entity={"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}} current payload={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}}],"actionType":"update"} | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 procesing j="type" attr=undefined ctxt={"c":1,"id":"motion001","type":"Thing","service":"smartcity","subservice":"/","entity_name":"Thing:motion001"} newAttr="Thing" | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 after procesing attr="type" current entity={"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}} current payload={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}}],"actionType":"update"} | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 procesing j="c" attr=undefined ctxt={"c":1,"id":"motion001","type":"Thing","service":"smartcity","subservice":"/","entity_name":"Thing:motion001"} newAttr={"value":"1","type":"string"} | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 after procesing attr="c" current entity={"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}} current payload={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string"}}],"actionType":"update"} | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 ending payload={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":"1","type":"string","metadata":{"TimeInstant":{"type":"DateTime","value":"2023-06-14T08:04:52.203Z"}}},"TimeInstant":{"type":"DateTime","value":"2023-06-14T08:04:52.203Z"}}],"actionType":"update"} | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=sendUpdateValueNgsi2 payload with native types and without object_id={"entities":[{"id":"Thing:motion001","type":"Thing","c":{"value":1,"type":"string","metadata":{"TimeInstant":{"type":"DateTime","value":"2023-06-14T08:04:52.203Z"}}},"TimeInstant":{"type":"DateTime","value":"2023-06-14T08:04:52.203Z"}}],"actionType":"update"} | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.NGSIUtils | from=n/a | srv=smartcity | subsrv=/ | msg=typeInformation {"lazy":null,"commands":[],"staticAttributes":[],"subscriptions":[],"_id":"648974a446e38536e98f93aa","creationDate":"2023-06-14T08:04:52.193Z","id":"motion001","type":"Thing","name":"Thing:motion001","service":"smartcity","subservice":"/","apikey":"4jggokgpepnvsb2uv4s40d59ov","protocol":"IoTA-UL","transport":"HTTP","v":0,"internalAttributes":null,"active":[]} | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=Updating device value in the Context Broker at [http://iot-orion:1026/v2/entities/Thing:motion001/attrs?type=Thing] | comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=Using the following NGSI v2 request:

{ "url": "http://iot-orion:1026/v2/entities/Thing:motion001/attrs?type=Thing", "method": "PATCH", "headers": { "fiware-service": "smartcity", "fiware-servicepath": "/" }, "json": { "c": { "value": 1, "type": "string", "metadata": { "TimeInstant": { "type": "DateTime", "value": "2023-06-14T08:04:52.203Z" } } }, "TimeInstant": { "type": "DateTime", "value": "2023-06-14T08:04:52.203Z" } } }

| comp=IoTAgent time=2023-06-14T08:04:52.203Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Request | from=n/a | srv=smartcity | subsrv=/ | msg=Options: { "url": "http://iot-orion:1026/v2/entities/Thing:motion001/attrs?type=Thing", "method": "PATCH", "headers": { "fiware-service": "smartcity", "fiware-servicepath": "/" }, "json": { "c": { "value": 1, "type": "string", "metadata": { "TimeInstant": { "type": "DateTime", "value": "2023-06-14T08:04:52.203Z" } } }, "TimeInstant": { "type": "DateTime", "value": "2023-06-14T08:04:52.203Z" } } } | comp=IoTAgent time=2023-06-14T08:04:52.207Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Request | from=n/a | srv=smartcity | subsrv=/ | msg=Response { "error": "Unprocessable", "description": "one or more of the attributes in the request do not exist: [ c ]" } | comp=IoTAgent time=2023-06-14T08:04:52.207Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IoTAgentNGSI.Entities-v2 | from=n/a | srv=smartcity | subsrv=/ | msg=Unknown error executing update operation | comp=IoTAgent time=2023-06-14T08:04:52.207Z | lvl=ERROR | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IOTAUL.HTTP.Binding | from=n/a | srv=smartcity | subsrv=/ | msg=MEASURES-002: Couldn't send the updated values to the Context Broker due to an error: {"name":"ENTITY_GENERIC_ERROR","message":"Error accesing entity data for device: Thing:motion001 of type: Thing","details":{"error":"Unprocessable","description":"one or more of the attributes in the request do not exist: [ c ]"},"code":422} | comp=IoTAgent time=2023-06-14T08:04:52.207Z | lvl=DEBUG | corr=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | trans=fced1dbc-df68-45c6-9fac-e22f8cecfd8d | op=IOTAUL.HTTP.Binding | from=n/a | srv=smartcity | subsrv=/ | msg=Error [ENTITY_GENERIC_ERROR] handing request: Error accesing entity data for device: Thing:motion001 of type: Thing | comp=IoTAgent

mapedraza commented 1 year ago

When using appendMode = true, no problem detected.

When using appendMode = false:

Creating a group with 1 attribute:

{
    "services": [
        {
            "resource": "/iot/json",
            "apikey": "80KL23Y9090V8733",
            "type": "Light",
            "attributes": [
                {
                    "object_id": "l",
                    "name": "luminescence",
                    "type": "Integer"
                }
            ]
        }
    ]
}

And then sending data with provisioned attributes 

{ "l": 10 }

No problem detected.

When sending data without provisioned attributes (this means any measure different from l)

422 Unprocessable Entity

{
    "name": "ENTITY_GENERIC_ERROR",
    "message": "Error accesing entity data for device: Light:dev2 of type: Light"
}

Device is created and entity in orion like the following 

{
        "id": "Light:dev3",
        "type": "Light",
        "TimeInstant": {
            "type": "DateTime",
            "value": "2023-06-14T09:08:15.080Z",
            "metadata": {}
        },
        "luminescence": {
            "type": "Integer",
            "value": null,
            "metadata": {}
        }
    }

As a summary:

This is the expected behaviour

Probably an additional note could be added to appendMode documentation section referencing error codes and messages

mapedraza commented 1 year ago

Added PR https://github.com/telefonicaid/iotagent-node-lib/pull/1382 to clarify this issue

KeshavSoni2511 commented 1 year ago

Hi @mapedraza , I think @gobaldia's issue was

Device was created when we send measurements through curl command API to a non existing device but for that there should be a proper Service apikey .

And the findings that I have shared was while the user has not created Service device was not autoprovisioned means the device with or without attributes and its value was not created.

I think you should try it with append mode = false and there should be a proper apikey and send the measurements to a non existing device to confirm whether the device is created or not with attributes or without attributes.

mapedraza commented 1 year ago

Hi @mapedraza , I think @gobaldia's issue was

Device was created when we send measurements through curl command API to a non existing device but for that there should be a proper Service apikey .

This is normal autoprovision behavior, everything seems ok

And the findings that I have shared was while the user has not created Service device was not autoprovisioned means the device with or without attributes and its value was not created.

Based on tests I performed, devices are created without issues (if autoprovision != false), no matter appendMode vale. Seems the expected behaviour

I think you should try it with append mode = false and there should be a proper apikey and send the measurements to a non existing device to confirm whether the device is created or not with attributes or without attributes.

Again, based on tests performed previously, using appendmode=false, the device and entity is created, but not updated since the entity does not contains all the attributes

IMHO this issue can be closed. Tests gave the expected behaviour. This issue is quite old and probably, if there was an issue, it has been solved

fgalan commented 1 year ago

IMHO this issue can be closed. Tests gave the expected behaviour. This issue is quite old and probably, if there was an issue, it has been solved

I agree. Closing.