Closed perezmlal closed 5 years ago
So the issue appears when you use commands array creating a device through the IoT Agent JSON
You mean that if you do the provision of the device using this
--data '{"devices":[{"device_id":"sensor001","entity_name":"sensor001","entity_type":"TestType","transport":"HTTP","protocol":"IoTA-JSON","endpoint":"http://simulator:3000/iot/test-type/sensor001","attributes":[{"object_id":"status","name":"status","type":"Text"}],"static_attributes":[{"name":"name","type":"Text","value":"Sensor 001"}]}]}'
Then it works as expected?
So the issue appears when you use commands array creating a device through the IoT Agent JSON
You mean that if you do the provision of the device using this
--data '{"devices":[{"device_id":"sensor001","entity_name":"sensor001","entity_type":"TestType","transport":"HTTP","protocol":"IoTA-JSON","endpoint":"http://simulator:3000/iot/test-type/sensor001","attributes":[{"object_id":"status","name":"status","type":"Text"}],"static_attributes":[{"name":"name","type":"Text","value":"Sensor 001"}]}]}'
Then it works as expected?
Yes, If you don't add commands array to that object, then it work as expected.
What about if you provision using the command array but you get the attribute individually, i.e.:
GET /v2/entities/sensor001/attrs/TimeInstant
GET /v2/entities/sensor001/attrs/example_attr
GET /v2/entities/sensor001/attrs/name
GET /v2/entities/sensor001/attrs/status
Is that working?
What about if you provision using the command array but you get the attribute individually, i.e.:
GET /v2/entities/sensor001/attrs/TimeInstant GET /v2/entities/sensor001/attrs/example_attr GET /v2/entities/sensor001/attrs/name GET /v2/entities/sensor001/attrs/status
Is that working?
Isn't working, the behaviour is the same.
I'll try to reproduce the problem. Could you post the docker-compose.yml you are using (reduced to the relevant container) so I can replay your same test, please?
I'll try to reproduce the problem. Could you post the docker-compose.yml you are using (reduced to the relevant container) so I can replay your same test, please?
Thanks @fgalan here you are.
version: "3.7"
services:
orion:
image: fiware/orion:2.0.0
container_name: orion
restart: always
depends_on:
- mongo-orion
ports:
- "1026:1026"
command: -corsOrigin __ALL -corsMaxAge 600 -dbhost mongo-orion
iotagent-json:
image: fiware/iotagent-json:1.8.0
container_name: iotagent-json
restart: always
depends_on:
- mongo-orion
- orion
ports:
- "4041:4041"
- "7896:7896"
environment:
- "IOTA_CB_HOST=orion"
- "IOTA_CB_PORT=1026"
- "IOTA_NORTH_PORT=4041"
- "IOTA_REGISTRY_TYPE=mongodb"
- "IOTA_LOG_LEVEL=DEBUG"
- "IOTA_TIMESTAMP=true"
- "IOTA_MONGO_HOST=mongo-orion"
- "IOTA_MONGO_PORT=27017"
- "IOTA_MONGO_DB=iotagent-json"
- "IOTA_HTTP_PORT=7896"
- "IOTA_PROVIDER_URL=http://iotagent-json:4041"
mongo-orion:
image: mongo:3.4
container_name: mongo-orion
restart: always
ports:
- "27017:27017"
For the records, an equivalente version of docker-compose.yml for version 1 (the one I have in the machine where I'm going to test).
orion:
image: fiware/orion:2.0.0
container_name: orion
restart: always
links:
- mongo-orion
ports:
- "1026:1026"
command: -corsOrigin __ALL -corsMaxAge 600 -dbhost mongo-orion
iotagent-json:
image: fiware/iotagent-json:1.8.0
container_name: iotagent-json
restart: always
links:
- mongo-orion
- orion
ports:
- "4041:4041"
- "7896:7896"
environment:
- "IOTA_CB_HOST=orion"
- "IOTA_CB_PORT=1026"
- "IOTA_NORTH_PORT=4041"
- "IOTA_REGISTRY_TYPE=mongodb"
- "IOTA_LOG_LEVEL=DEBUG"
- "IOTA_TIMESTAMP=true"
- "IOTA_MONGO_HOST=mongo-orion"
- "IOTA_MONGO_PORT=27017"
- "IOTA_MONGO_DB=iotagent-json"
- "IOTA_HTTP_PORT=7896"
- "IOTA_PROVIDER_URL=http://iotagent-json:4041"
mongo-orion:
image: mongo:3.4
container_name: mongo-orion
restart: always
ports:
- "27017:27017"
Hint: using this for CB command
you will bet a more useful log:
command: -corsOrigin __ALL -corsMaxAge 600 -dbhost mongo-orion -logLevel DEBUG -logForHumans -noCache -t 180-199
I'm doing the same test, but getting a different result:
$ curl -s -S --request GET --url http://localhost:1026/v2/entities/sensor001 --header 'fiware-service: testService' --header 'fiware-servicepath: /testPath' | python -mjson.tool
{
"TimeInstant": {
"metadata": {},
"type": "ISO8601",
"value": " "
},
"example_attr": {
"metadata": {},
"type": "StructuredValue"
},
"id": "sensor001",
"name": {
"metadata": {},
"type": "Text",
"value": "Sensor 001"
},
"ping_info": {
"metadata": {},
"type": "commandResult",
"value": " "
},
"ping_status": {
"metadata": {},
"type": "commandStatus",
"value": "UNKNOWN"
},
"status": {
"metadata": {},
"type": "Text",
"value": " "
},
"type": "TestType"
}
So it it working in almost all cases... except in the one associated to a structured value. For that case, the value
key is not shown.
The log I get in Orion is the following one:
orion | INFO@14:55:49 logMsg.h[1832]: Starting transaction from 172.17.42.1:35657/v2/entities/sensor001
orion | INFO@14:55:49 rest.cpp[885]: Service Path 0: '/testPath'
orion | INFO@14:55:49 connectionOperations.cpp[177]: Database Operation Successful (query: { query: { $or: [ { _id.id: "sensor001" } ], _id.servicePath: { $in: [ /^/testPath$/ ] } }, orderby: { creDate: 1 } })
orion | INFO@14:55:49 connectionOperations.cpp[177]: Database Operation Successful (query: { query: { $or: [ { contextRegistration.entities: { $in: [] } }, { contextRegistration.entities.id: { $in: [ "sensor001" ] } } ], expiration: { $gt: 1544021749 }, servicePath: { $in: [ /^/testPath$/ ] } }, orderby: { _id: 1 } })
orion | DEBUG@14:55:49 postQueryContext.cpp[123]: forward queryContext request payload: {"entities":[{"type":"TestType","isPattern":"false","id":"sensor001"}],"attributes":["ping"]}
orion | INFO@14:55:49 logMsg.h[1832]: Starting transaction to http://iotagent-json:4041//queryContext
orion | INFO@14:55:49 httpRequestSend.cpp[594]: Sending message 6 to HTTP server: sending message of 362 bytes to HTTP server
orion | INFO@14:55:50 logMsg.h[1916]: Transaction ended
orion | WARN@14:55:50 postQueryContext.cpp[146]: Runtime Error (error forwarding 'Query' to providing application)
orion | ERROR@14:55:50 ContextAttribute.cpp[940]: Runtime Error (invalid value type for attribute example_attr)
orion | DEBUG@14:55:50 restReply.cpp[75]: Response 9: responding with 394 bytes, Status Code 200
orion | DEBUG@14:55:50 restReply.cpp[76]: Response payload: '{"id":"sensor001","type":"TestType","TimeInstant":{"type":"ISO8601","value":" ","metadata":{}},"example_attr":{"type":"StructuredValue","metadata":{}},"name":{"type":"Text","value":"Sensor 001","metadata":{}},"ping_info":{"type":"commandResult","value":" ","metadata":{}},"ping_status":{"type":"commandStatus","value":"UNKNOWN","metadata":{}},"status":{"type":"Text","value":" ","metadata":{}}}'
orion | INFO@14:55:50 logMsg.h[1916]: Transaction ended
The lack of iota-json log in this case make me think that probably the Orion->IOTA-JSON connection is not working. I'll have a look to that next...
I have confirmed the connection problems. Looking to the IOTA-JSON container IP:
$ docker inspect <iota_json>| grep "IPAddress"
"SecondaryIPAddresses": null,
"IPAddress": "172.17.0.4",
"IPAddress": "172.17.0.4",
Now accessing to Orion container:
$ docker exec -ti <orion_docker> bash
Trying to ping by name:
[root@0917907dbcda /]# ping iotagent-json
ping: iotagent-json: Name or service not known
Thus, I'm adding "172.17.0.4 iotagent-json" to /etc/hosts and now it works
[root@0917907dbcda /]# ping iotagent-json
PING iotagent-json (127.17.0.4) 56(84) bytes of data.
64 bytes from iotagent-json (127.17.0.4): icmp_seq=1 ttl=64 time=0.050 ms
64 bytes from iotagent-json (127.17.0.4): icmp_seq=2 ttl=64 time=0.131 ms
^C
--- iotagent-json ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
Running again the above query:
$ curl -s -S --request GET --url http://localhost:1026/v2/entities/sensor001 --header 'fiware-service: testService' --header 'fiware-servicepath: /testPath' | python -mjson.tool
{
"TimeInstant": {
"metadata": {},
"type": "ISO8601",
"value": " "
},
"example_attr": {
"metadata": {},
"type": "StructuredValue"
},
"id": "sensor001",
"name": {
"metadata": {},
"type": "Text",
"value": "Sensor 001"
},
"ping_info": {
"metadata": {},
"type": "commandResult",
"value": " "
},
"ping_status": {
"metadata": {},
"type": "commandStatus",
"value": "UNKNOWN"
},
"status": {
"metadata": {},
"type": "Text",
"value": " "
},
"type": "TestType"
}
Log:
orion | INFO@15:09:58 logMsg.h[1832]: Starting transaction from 172.17.42.1:36640/v2/entities/sensor001
orion | INFO@15:09:58 rest.cpp[885]: Service Path 0: '/testPath'
orion | INFO@15:09:58 connectionOperations.cpp[177]: Database Operation Successful (query: { query: { $or: [ { _id.id: "sensor001" } ], _id.servicePath: { $in: [ /^/testPath$/ ] } }, orderby: { creDate: 1 } })
orion | INFO@15:09:58 connectionOperations.cpp[177]: Database Operation Successful (query: { query: { $or: [ { contextRegistration.entities: { $in: [] } }, { contextRegistration.entities.id: { $in: [ "sensor001" ] } } ], expiration: { $gt: 1544022598 }, servicePath: { $in: [ /^/testPath$/ ] } }, orderby: { _id: 1 } })
orion | DEBUG@15:09:58 postQueryContext.cpp[123]: forward queryContext request payload: {"entities":[{"type":"TestType","isPattern":"false","id":"sensor001"}],"attributes":["ping"]}
orion | INFO@15:09:58 logMsg.h[1832]: Starting transaction to http://iotagent-json:4041//queryContext
orion | INFO@15:09:58 httpRequestSend.cpp[594]: Sending message 9 to HTTP server: sending message of 362 bytes to HTTP server
iotagent-json | time=2018-12-05T15:09:58.078Z | lvl=DEBUG | corr=d454b4c2-f89f-11e8-9c7c-0242ac110003 | trans=00e98208-3309-4260-8a98-169c49dec2fb | op=IoTAgentNGSI.GenericMiddlewares | srv=testservice | subsrv=/testPath | msg=Request for path [//queryContext] from [iotagent-json:4041] | comp=IoTAgent
iotagent-json | time=2018-12-05T15:09:58.078Z | lvl=DEBUG | corr=d454b4c2-f89f-11e8-9c7c-0242ac110003 | trans=00e98208-3309-4260-8a98-169c49dec2fb | op=IoTAgentNGSI.GenericMiddlewares | srv=testservice | subsrv=/testPath | msg=Body:
iotagent-json |
iotagent-json | {
iotagent-json | "entities": [
iotagent-json | {
iotagent-json | "type": "TestType",
iotagent-json | "isPattern": "false",
iotagent-json | "id": "sensor001"
iotagent-json | }
iotagent-json | ],
iotagent-json | "attributes": [
iotagent-json | "ping"
iotagent-json | ]
iotagent-json | }
iotagent-json |
iotagent-json | | comp=IoTAgent
iotagent-json | time=2018-12-05T15:09:58.084Z | lvl=DEBUG | corr=d454b4c2-f89f-11e8-9c7c-0242ac110003 | trans=00e98208-3309-4260-8a98-169c49dec2fb | op=IoTAgentNGSI.ContextServer | srv=testservice | subsrv=/testPath | msg=Handling query from [iotagent-json:4041] | comp=IoTAgent
iotagent-json | time=2018-12-05T15:09:58.087Z | lvl=DEBUG | corr=d454b4c2-f89f-11e8-9c7c-0242ac110003 | trans=00e98208-3309-4260-8a98-169c49dec2fb | op=IoTAgentNGSI.MongoDBDeviceRegister | srv=testservice | subsrv=/testPath | msg=Looking for device with name [sensor001]. | comp=IoTAgent
iotagent-json | time=2018-12-05T15:09:58.110Z | lvl=DEBUG | corr=d454b4c2-f89f-11e8-9c7c-0242ac110003 | trans=00e98208-3309-4260-8a98-169c49dec2fb | op=IoTAgentNGSI.MongoDBDeviceRegister | srv=testservice | subsrv=/testPath | msg=Device [sensor001] not found. | comp=IoTAgent
iotagent-json | time=2018-12-05T15:09:58.110Z | lvl=ERROR | corr=d454b4c2-f89f-11e8-9c7c-0242ac110003 | trans=00e98208-3309-4260-8a98-169c49dec2fb | op=IoTAgentNGSI.Alarms | srv=testservice | subsrv=/testPath | msg=Raising [MONGO-ALARM]: {"name":"DEVICE_NOT_FOUND","message":"No device was found with id:sensor001","code":404} | comp=IoTAgent
iotagent-json | time=2018-12-05T15:09:58.111Z | lvl=DEBUG | corr=d454b4c2-f89f-11e8-9c7c-0242ac110003 | trans=00e98208-3309-4260-8a98-169c49dec2fb | op=IoTAgentNGSI.ContextServer | srv=testservice | subsrv=/testPath | msg=There was an error handling the query: [object Object]. | comp=IoTAgent
iotagent-json | time=2018-12-05T15:09:58.111Z | lvl=DEBUG | corr=d454b4c2-f89f-11e8-9c7c-0242ac110003 | trans=00e98208-3309-4260-8a98-169c49dec2fb | op=IoTAgentNGSI.ContextServer | srv=testservice | subsrv=/testPath | msg=Query error [DEVICE_NOT_FOUND] handling request: No device was found with id:sensor001 | comp=IoTAgent
orion | INFO@15:09:58 httpRequestSend.cpp[615]: Notification Successfully Sent to http://iotagent-json:4041//queryContext
orion | INFO@15:09:58 logMsg.h[1916]: Transaction ended
orion | DEBUG@15:09:58 postQueryContext.cpp[150]: forward queryContext response payload: HTTP/1.1 404 Not Found
orion | X-Powered-By: Express
orion | Fiware-Correlator: d454b4c2-f89f-11e8-9c7c-0242ac110003
orion | Content-Type: application/json; charset=utf-8
orion | Content-Length: 110
orion | Date: Wed, 05 Dec 2018 15:09:58 GMT
orion | Connection: keep-alive
orion |
orion | {"errorCode":{"code":404,"reasonPhrase":"DEVICE_NOT_FOUND","details":"No device was found with id:sensor001"}}
orion | ERROR@15:09:58 ContextAttribute.cpp[940]: Runtime Error (invalid value type for attribute example_attr)
orion | DEBUG@15:09:58 restReply.cpp[75]: Response 12: responding with 394 bytes, Status Code 200
iotagent-json | time=2018-12-05T15:09:58.114Z | lvl=DEBUG | corr=d454b4c2-f89f-11e8-9c7c-0242ac110003 | trans=00e98208-3309-4260-8a98-169c49dec2fb | op=IoTAgentNGSI.DomainControl | srv=testservice | subsrv=/testPath | msg=response-time: 38 | comp=IoTAgent
orion | DEBUG@15:09:58 restReply.cpp[76]: Response payload: '{"id":"sensor001","type":"TestType","TimeInstant":{"type":"ISO8601","value":" ","metadata":{}},"example_attr":{"type":"StructuredValue","metadata":{}},"name":{"type":"Text","value":"Sensor 001","metadata":{}},"ping_info":{"type":"commandResult","value":" ","metadata":{}},"ping_status":{"type":"commandStatus","value":"UNKNOWN","metadata":{}},"status":{"type":"Text","value":" ","metadata":{}}}'
orion | INFO@15:09:58 logMsg.h[1916]: Transaction ended
I'm getting the same result but now the log shows Orion-IOTA-JSON interaction. The problem is not yet solved, but having that ok is a necesary condition to have the whole system to work. Maybe something is missing in docker-compose.yml so the name binding for IOTA-JSON gets created inside the Orion container.
Let's analyze this iteration at IOTA:
Context Broker forwards the query to IOTA (honouring the registration for the ping attribute):
time=2018-12-05T15:09:58.078Z | lvl=DEBUG | corr=d454b4c2-f89f-11e8-9c7c-0242ac110003 | trans=00e98208-3309-4260-8a98-169c49dec2fb | op=IoTAgentNGSI.GenericMiddlewares | srv=testservice | subsrv=/testPath | msg=Body:
iotagent-json |
iotagent-json | {
iotagent-json | "entities": [
iotagent-json | {
iotagent-json | "type": "TestType",
iotagent-json | "isPattern": "false",
iotagent-json | "id": "sensor001"
iotagent-json | }
iotagent-json | ],
iotagent-json | "attributes": [
iotagent-json | "ping"
iotagent-json | ]
iotagent-json | }
and IOTA responses:
orion | DEBUG@15:09:58 postQueryContext.cpp[150]: forward queryContext response payload: HTTP/1.1 404 Not Found
orion | X-Powered-By: Express
orion | Fiware-Correlator: d454b4c2-f89f-11e8-9c7c-0242ac110003
orion | Content-Type: application/json; charset=utf-8
orion | Content-Length: 110
orion | Date: Wed, 05 Dec 2018 15:09:58 GMT
orion | Connection: keep-alive
orion |
orion | {"errorCode":{"code":404,"reasonPhrase":"DEVICE_NOT_FOUND","details":"No device was found with id:sensor001"}}
Is correct to responde with a 404 error here? I think so, as the IOTA is not prepared to process queries on command attributes (like ping). It is prepared to process updates on command attributes or queries on lazy attributes (at least in theory, as lazy attributes support is yet pending in that agent, see https://github.com/telefonicaid/iotagent-json/issues/89).
However, If my guessing is correct, what is not so correct is the reasonPhrase, as sensor001 do exists.
After this debugging session, it seems we have two situations here:
orion | WARN@14:55:50 postQueryContext.cpp[146]: Runtime Error (error forwarding 'Query' to providing application)
orion | ERROR@14:55:50 ContextAttribute.cpp[940]: Runtime Error (invalid value type for attribute example_attr)
orion | ERROR@15:09:58 ContextAttribute.cpp[940]: Runtime Error (invalid value type for attribute example_attr)
In both situations, the value
key for compound attributes is not rendered in the query result. Other attributes (not compound) seems to work fine.
Once the problem has been narrowed down, the next step is trying to create a .test able to reproduce the case isolately and debug it.
This issue could be similar to https://github.com/telefonicaid/fiware-orion/issues/3162 although I'm not fully sure, as that issue in that case is seems that the value
key is included but with a blank (" "
) value. It would be a good idea to have a new look to #3162 once this issue gets solved.
Reference documents in orion-testservice DB (useful to implement the .test):
> db.registrations.find().pretty()
{
"_id" : ObjectId("5c0f5b23a143e165b0408cb1"),
"expiration" : NumberLong(1547102243),
"servicePath" : "/testPath",
"format" : "JSON",
"contextRegistration" : [
{
"entities" : [
{
"id" : "sensor001",
"type" : "TestType"
}
],
"attrs" : [
{
"name" : "ping",
"type" : "command"
}
],
"providingApplication" : "http://iotagent-json:4041"
}
]
}
> db.entities.find().pretty()
{
"_id" : {
"id" : "sensor001",
"type" : "TestType",
"servicePath" : "/testPath"
},
"attrNames" : [
"status",
"name",
"ping_status",
"ping_info",
"TimeInstant",
"example_attr"
],
"attrs" : {
"status" : {
"type" : "Text",
"creDate" : 1544510243,
"modDate" : 1544510243,
"value" : " ",
"mdNames" : [ ]
},
"name" : {
"type" : "Text",
"creDate" : 1544510243,
"modDate" : 1544510243,
"value" : "Sensor 001",
"mdNames" : [ ]
},
"ping_status" : {
"type" : "commandStatus",
"creDate" : 1544510243,
"modDate" : 1544510243,
"value" : "UNKNOWN",
"mdNames" : [ ]
},
"ping_info" : {
"type" : "commandResult",
"creDate" : 1544510243,
"modDate" : 1544510243,
"value" : " ",
"mdNames" : [ ]
},
"TimeInstant" : {
"type" : "ISO8601",
"creDate" : 1544510243,
"modDate" : 1544510243,
"value" : " ",
"mdNames" : [ ]
},
"example_attr" : {
"value" : {
"exampleField" : "exampleValue"
},
"type" : "StructuredValue",
"mdNames" : [ ],
"creDate" : 1544510256,
"modDate" : 1544510256
}
},
"creDate" : 1544510243,
"modDate" : 1544510256,
"lastCorrelator" : "3f6915be-fd0f-11e8-88a6-0242ac110003"
}
The fix has landed in master branch. Please @tioperez redeploy using fiware/orion:latest
(instead of :2.0.0
) and check if example_attr
is working now. Thanks!
Thanks @fgalan
You're welcome!
"Thanks" means that you have tested and it is working now? :)
Yeap works fine 👌
Thanks for the feedback! Based in your report I think we can close this issue.
I've deployed the iot stack with docker. These are the used containers:
[other containers not involved in the issue ...]
Documentation consulted:
GIVEN: A device ready to receive a command called "ping"
Creating that Device through the IoT Agent JSON
WHEN: Update or append to that entity now in Orion CB an attribute of type StructuredValue
THEN: Retrieve an entity by ID sensor001 or List entities
The response does not contains the value for the attribute example_attr
So the issue appears when you use commands array creating a device through the IoT Agent JSON