orchestracities / ngsi-timeseries-api

QuantumLeap: a FIWARE Generic Enabler to support the usage of NGSIv2 (and NGSI-LD experimentally) data in time-series databases
https://quantumleap.rtfd.io/
MIT License
38 stars 49 forks source link

QuantumLeap integration with OrionCB and IoTagent-LoRaWAN #329

Closed jfernandz closed 4 years ago

jfernandz commented 4 years ago

Describe the bug I'm not sure if this is a bug or I am doing something wrong.

To Reproduce Steps to reproduce the behavior:

  1. First I've posted a device in the IoTagent-LoRaWAN and apparently is receiving the payload as you can see in the picture Anotación 2020-06-24 215108
  2. However I'm gettin that "level":"error","message":"Could not cast message to NGSI" error message in the logs despite all containers are working as expected. Also I'm not sure how OrionCB, IoTagent-LoRaWAN and QuantumLeap interact between them; according to their docs I have to create a subscription in order to OrionCB notifies QuantumLeap to store the information in CrateDB but ... Should not I first create an entity attached to the IoTagent-LoRaWAN device that I've previously created?

Expected behavior POST request to http://localhost:8668/v2/subscribe?orionUrl=http://localhost:1026/v2&quantumleapUrl=http://localhost:8668/v2 returns a 200 OK code (?)

Environment (please complete the following information):

Additional context I'll provide in followin answers if necessary

Sorry if this is not the right place to ask this kind of things but I did also a question in StackOverflow but I'm not sure this will be answered

jfernandz commented 4 years ago

Well, apparently it was my fault and the docker-compose.yml file was not deploying properly mongodb and orioncb so they cannot connect between them. However, ... now I'm able to create an entity through a POST request to the IoTagent-LoRaWAN when I create a device, also I'm able to create the subscription sending this body in another POST request to orioncb

{
  "description": "A subscription to get info about LORA-N-0",
  "subject": {
    "entities": [
      {
        "id": "LORA-N-0",
        "type": "LoraDevice"
      }
    ],
    "condition": {
      "attrs": [
        "analog_in_1"
      ]
    }
  },
  "notification": {
    "http": {
      "url": "http://quantumleap:8668/v2/notify"
    },
    "attrs": [
      "analog_in_1"
    ],
    "metadata": ["dateCreated", "dateModified"]
  },
  "throttling": 5
}

As you can see in the picture the subscription is apparently created correctly Captura de pantalla_2020-06-25_11-15-55

However ... also as you can see in the picture ... QuantumLeap is throwing the following exception with its corresponding traceback

quantumleap_1  | INFO:reporter.reporter:Received payload: {'id': 'LORA-N-0', 'type': 'LoraDevice', 'analog_in_1': {'type': 'Number', 'value': 96.64, 'metadata': {'dateCreated': {'type': 'DateTime', 'value': '2020-06-25T08:36:55.00Z'}, 'dateModified': {'type': 'DateTime', 'value': '2020-06-25T09:21:41.00Z'}}}}
quantumleap_1  | INFO:geocoding.geocoding:Cannot add location to entity (type: "LoraDevice", id: "LORA-N-0"), missing "address" attribute.
quantumleap_1  | [2020-06-25 09:21:41,652] ERROR in app: Exception on /v2/notify [POST]
quantumleap_1  | Traceback (most recent call last):
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
quantumleap_1  |     response = self.full_dispatch_request()
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
quantumleap_1  |     rv = self.handle_user_exception(e)
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
quantumleap_1  |     reraise(exc_type, exc_value, tb)
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
quantumleap_1  |     raise value
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
quantumleap_1  |     rv = self.dispatch_request()
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
quantumleap_1  |     return self.view_functions[rule.endpoint](**req.view_args)
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/connexion/decorators/decorator.py", line 73, in wrapper
quantumleap_1  |     response = function(request)
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/connexion/decorators/uri_parsing.py", line 117, in wrapper
quantumleap_1  |     response = function(request)
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/connexion/decorators/validation.py", line 163, in wrapper
quantumleap_1  |     response = function(request)
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/connexion/decorators/validation.py", line 336, in wrapper
quantumleap_1  |     return function(request)
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/connexion/decorators/decorator.py", line 44, in wrapper
quantumleap_1  |     response = function(request)
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/connexion/decorators/parameter.py", line 207, in wrapper
quantumleap_1  |     return function(**kwargs)
quantumleap_1  |   File "/src/ngsi-timeseries-api/src/reporter/reporter.py", line 179, in notify
quantumleap_1  |     trans.insert([payload], fiware_s, fiware_sp)
quantumleap_1  |   File "/src/ngsi-timeseries-api/src/translators/crate.py", line 183, in insert
quantumleap_1  |     fiware_servicepath)
quantumleap_1  |   File "/src/ngsi-timeseries-api/src/translators/crate.py", line 260, in _insert_entities_of_type
quantumleap_1  |     self._update_metadata_table(table_name, original_attrs)
quantumleap_1  |   File "/src/ngsi-timeseries-api/src/translators/crate.py", line 352, in _update_metadata_table
quantumleap_1  |     self.cursor.execute(stmt, (table_name, persisted_metadata))
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/crate/client/cursor.py", line 54, in execute
quantumleap_1  |     bulk_parameters)
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/crate/client/http.py", line 328, in sql
quantumleap_1  |     content = self._json_request('POST', self.path, data=data)
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/crate/client/http.py", line 448, in _json_request
quantumleap_1  |     _raise_for_status(response)
quantumleap_1  |   File "/usr/local/lib/python3.6/site-packages/crate/client/http.py", line 187, in _raise_for_status
quantumleap_1  |     error_trace=error_trace)
quantumleap_1  | crate.client.exceptions.ProgrammingError: SQLActionException[SQLParseException: line 1:72: no viable alternative at input 'on duplicate']
quantumleap_1  | 172.24.0.6 - - [25/Jun/2020 09:21:41] "POST /v2/notify HTTP/1.1" 500 -
quantumleap_1  | INFO:werkzeug:172.24.0.6 - - [25/Jun/2020 09:21:41] "POST /v2/notify HTTP/1.1" 500 -

Here is also my docker-compose.yml Is it maybe the QuantumLeap version? :thinking:

I don't know if I'm doing something wrong or this is caused by QuantumLeap itself but ... I don't like that

quantumleap_1 | [2020-06-25 09:21:41,652] ERROR in app: Exception on /v2/notify [POST]

What do you think?

c0c0n3 commented 4 years ago

Hello @WyRe, thank you for getting in touch.

POST request to http://localhost:8668/v2/subscribe?orionUrl=http://localhost:1026/v2&quantumleapUrl=http://localhost:8668/v2 returns a 200 OK code (?)

As you noted, the problem there was that in your initial setup QuantumLeap wouldn't be able to reach Orion on localhost since Orion was in the Docker network. If you'd like to access QuantumLeap and Orion from your localhost too, you could try setting up a Docker compose bridge network, see e.g. https://github.com/smartsdk/ngsi-timeseries-api/blob/master/src/reporter/tests/docker-compose.yml.

Here is also my docker-compose.yml Is it maybe the QuantumLeap version?

Most likely yes. The version you have in your Docker compose (0.5) doesn't work with the latest Crate---the crate image points to version 4.1 which isn't supported by QuantumLeap 0.5. Try changing your Docker compose like so

quantumleap:
    image: smartsdk/quantumleap:0.7.5
...
crate:
    image: crate:3.3.5
...

Hope this helps.

jfernandz commented 4 years ago

@c0c0n3 oh no, thank to you! Now it's apparently working :smile:

fiware-iot-agent | time=2020-06-25T10:05:58.790Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=error {"name":"DEVICE_GROUP_NOT_FOUND","message":"Couldn\t find device group","code":404} in get group device | comp=IoTAgent
fiware-iot-agent | time=2020-06-25T10:05:58.791Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=typeInformation {"id":"node_0","type":"LoraDevice","name":"LORA-N-0","service":"atosioe","subservice":"/lorattn","active":[{"object_id":"analog_in_1","name":"analog_in_1","type":"Number"}],"timezone":"Europe/Madrid","internalAttributes":{"lorawan":{"application_server":{"host":"eu.thethings.network","username":"dendrometer","password":"ttn-account-v2.173lH8wwDiIRC8E2JgM9ScXyuNRlPpMefpazS0TIhnU","provider":"TTN"},"dev_eui":"00EC51D264A6F8FD","app_eui":"70B3D57ED0030B5D","application_id":"dendrometer","application_key":"7AF6B9A29EAF893F66C0F4360BA8DD5B"}},"internalId":null,"subscriptions":[],"creationDate":1593079329781} | comp=IoTAgent
fiware-iot-agent | time=2020-06-25T10:05:58.791Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=typeInformation: {"id":"node_0","type":"LoraDevice","name":"LORA-N-0","service":"atosioe","subservice":"/lorattn","active":[{"object_id":"analog_in_1","name":"analog_in_1","type":"Number"}],"timezone":"Europe/Madrid","internalAttributes":{"lorawan":{"application_server":{"host":"eu.thethings.network","username":"dendrometer","password":"ttn-account-v2.173lH8wwDiIRC8E2JgM9ScXyuNRlPpMefpazS0TIhnU","provider":"TTN"},"dev_eui":"00EC51D264A6F8FD","app_eui":"70B3D57ED0030B5D","application_id":"dendrometer","application_key":"7AF6B9A29EAF893F66C0F4360BA8DD5B"}},"internalId":null,"subscriptions":[],"creationDate":1593079329781} | comp=IoTAgent
fiware-iot-agent | time=2020-06-25T10:05:58.792Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=Updating device value in the Context Broker at [http://orion:1026/v2/entities/LORA-N-0/attrs?type=LoraDevice] | comp=IoTAgent
fiware-iot-agent | time=2020-06-25T10:05:58.792Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=Using the following request:
fiware-iot-agent | {
fiware-iot-agent |     "url": "http://orion:1026/v2/entities/LORA-N-0/attrs?type=LoraDevice",
fiware-iot-agent |     "method": "POST",
fiware-iot-agent |     "headers": {
fiware-iot-agent |         "fiware-service": "atosioe",
fiware-iot-agent |         "fiware-servicepath": "/lorattn"
fiware-iot-agent |     },
fiware-iot-agent |     "json": {
fiware-iot-agent |         "analog_in_1": {
fiware-iot-agent |             "type": "Number",
fiware-iot-agent |             "value": 96.64,
fiware-iot-agent |             "metadata": {
fiware-iot-agent |                 "TimeInstant": {
fiware-iot-agent |                     "type": "DateTime",
fiware-iot-agent |                     "value": "2020-06-25T12:05:58.791+02:00"
fiware-iot-agent |                 }
fiware-iot-agent |             }
fiware-iot-agent |         },
fiware-iot-agent |         "TimeInstant": {
fiware-iot-agent |             "type": "DateTime",
fiware-iot-agent |             "value": "2020-06-25T12:05:58.791+02:00"
fiware-iot-agent |         }
fiware-iot-agent |     }
fiware-iot-agent | }
fiware-iot-agent |  | comp=IoTAgent
fiware-iot-agent | time=2020-06-25T10:05:58.801Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=Received the following response from the CB: Value updated successfully
fiware-iot-agent |  | comp=IoTAgent
fiware-iot-agent | {"timestamp":"2020-06-25T10:05:58.801Z","level":"info","message":"Observations sent to CB successfully for device "}
fiware-iot-agent | {"timestamp":"2020-06-25T10:07:03.458Z","level":"info","message":"New message in topic"}
fiware-iot-agent | {"timestamp":"2020-06-25T10:07:03.459Z","level":"info","message":"IOTA provisioned devices:"}
fiware-iot-agent | {"timestamp":"2020-06-25T10:07:03.460Z","level":"info","message":"Decoding CaynneLPP message:AQIlwA=="}
fiware-iot-agent | time=2020-06-25T10:07:03.460Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=executeWithDeviceInfo entityName LORA-N-0 type LoraDevice apikey  attributes [{"name":"analog_in_1","type":"Number","value":96.64}] deviceInformation {"id":"node_0","type":"LoraDevice","name":"LORA-N-0","service":"atosioe","subservice":"/lorattn","active":[{"object_id":"analog_in_1","name":"analog_in_1","type":"Number"}],"timezone":"Europe/Madrid","internalAttributes":{"lorawan":{"application_server":{"host":"eu.thethings.network","username":"dendrometer","password":"ttn-account-v2.173lH8wwDiIRC8E2JgM9ScXyuNRlPpMefpazS0TIhnU","provider":"TTN"},"dev_eui":"00EC51D264A6F8FD","app_eui":"70B3D57ED0030B5D","application_id":"dendrometer","application_key":"7AF6B9A29EAF893F66C0F4360BA8DD5B"}},"internalId":null,"subscriptions":[],"creationDate":1593079329781} | comp=IoTAgent
fiware-iot-agent | time=2020-06-25T10:07:03.461Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=error {"name":"DEVICE_GROUP_NOT_FOUND","message":"Couldn\t find device group","code":404} in get group device | comp=IoTAgent
fiware-iot-agent | time=2020-06-25T10:07:03.461Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=typeInformation {"id":"node_0","type":"LoraDevice","name":"LORA-N-0","service":"atosioe","subservice":"/lorattn","active":[{"object_id":"analog_in_1","name":"analog_in_1","type":"Number"}],"timezone":"Europe/Madrid","internalAttributes":{"lorawan":{"application_server":{"host":"eu.thethings.network","username":"dendrometer","password":"ttn-account-v2.173lH8wwDiIRC8E2JgM9ScXyuNRlPpMefpazS0TIhnU","provider":"TTN"},"dev_eui":"00EC51D264A6F8FD","app_eui":"70B3D57ED0030B5D","application_id":"dendrometer","application_key":"7AF6B9A29EAF893F66C0F4360BA8DD5B"}},"internalId":null,"subscriptions":[],"creationDate":1593079329781} | comp=IoTAgent
fiware-iot-agent | time=2020-06-25T10:07:03.468Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=typeInformation: {"id":"node_0","type":"LoraDevice","name":"LORA-N-0","service":"atosioe","subservice":"/lorattn","active":[{"object_id":"analog_in_1","name":"analog_in_1","type":"Number"}],"timezone":"Europe/Madrid","internalAttributes":{"lorawan":{"application_server":{"host":"eu.thethings.network","username":"dendrometer","password":"ttn-account-v2.173lH8wwDiIRC8E2JgM9ScXyuNRlPpMefpazS0TIhnU","provider":"TTN"},"dev_eui":"00EC51D264A6F8FD","app_eui":"70B3D57ED0030B5D","application_id":"dendrometer","application_key":"7AF6B9A29EAF893F66C0F4360BA8DD5B"}},"internalId":null,"subscriptions":[],"creationDate":1593079329781} | comp=IoTAgent
fiware-iot-agent | time=2020-06-25T10:07:03.470Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=Updating device value in the Context Broker at [http://orion:1026/v2/entities/LORA-N-0/attrs?type=LoraDevice] | comp=IoTAgent
fiware-iot-agent | time=2020-06-25T10:07:03.470Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=Using the following request:
fiware-iot-agent | {
fiware-iot-agent |     "url": "http://orion:1026/v2/entities/LORA-N-0/attrs?type=LoraDevice",
fiware-iot-agent |     "method": "POST",
fiware-iot-agent |     "headers": {
fiware-iot-agent |         "fiware-service": "atosioe",
fiware-iot-agent |         "fiware-servicepath": "/lorattn"
fiware-iot-agent |     },
fiware-iot-agent |     "json": {
fiware-iot-agent |         "analog_in_1": {
fiware-iot-agent |             "type": "Number",
fiware-iot-agent |             "value": 96.64,
fiware-iot-agent |             "metadata": {
fiware-iot-agent |                 "TimeInstant": {
fiware-iot-agent |                     "type": "DateTime",
fiware-iot-agent |                     "value": "2020-06-25T12:07:03.469+02:00"
fiware-iot-agent |                 }
fiware-iot-agent |             }
fiware-iot-agent |         },
fiware-iot-agent |         "TimeInstant": {
fiware-iot-agent |             "type": "DateTime",
fiware-iot-agent |             "value": "2020-06-25T12:07:03.469+02:00"
fiware-iot-agent |         }
fiware-iot-agent |     }
fiware-iot-agent | }
fiware-iot-agent |  | comp=IoTAgent
fiware-orion   | time=Thursday 25 Jun 10:07:03 2020.475Z | lvl=INFO | corr=9e10fc36-b6cb-11ea-9471-0242ac1a0005 | trans=1593079220-399-00000000014 | from=172.26.0.6 | srv=atosioe | subsrv=/lorattn | comp=Orion | op=logMsg.h[1844]:lmTransactionStart | msg=Starting transaction from 172.26.0.6:50330/v2/entities/LORA-N-0/attrs
fiware-orion   | time=Thursday 25 Jun 10:07:03 2020.476Z | lvl=INFO | corr=9e10fc36-b6cb-11ea-9471-0242ac1a0005 | trans=1593079220-399-00000000014 | from=172.26.0.6 | srv=atosioe | subsrv=/lorattn | comp=Orion | op=rest.cpp[874]:servicePathSplit | msg=Service Path 0: '/lorattn'
fiware-orion   | time=Thursday 25 Jun 10:07:03 2020.477Z | lvl=INFO | corr=9e10fc36-b6cb-11ea-9471-0242ac1a0005 | trans=1593079220-399-00000000014 | from=172.26.0.6 | srv=atosioe | subsrv=/lorattn | comp=Orion | op=connectionOperations.cpp[244]:collectionCount | msg=Database Operation Successful (count: { _id.id: "LORA-N-0", _id.type: "LoraDevice", _id.servicePath: "/lorattn" })
fiware-orion   | time=Thursday 25 Jun 10:07:03 2020.478Z | lvl=INFO | corr=9e10fc36-b6cb-11ea-9471-0242ac1a0005 | trans=1593079220-399-00000000014 | from=172.26.0.6 | srv=atosioe | subsrv=/lorattn | comp=Orion | op=connectionOperations.cpp[94]:collectionQuery | msg=Database Operation Successful (query: { _id.id: "LORA-N-0", _id.type: "LoraDevice", _id.servicePath: "/lorattn" })
fiware-orion   | time=Thursday 25 Jun 10:07:03 2020.479Z | lvl=INFO | corr=9e10fc36-b6cb-11ea-9471-0242ac1a0005 | trans=1593079220-399-00000000014 | from=172.26.0.6 | srv=atosioe | subsrv=/lorattn | comp=Orion | op=connectionOperations.cpp[454]:collectionUpdate | msg=Database Operation Successful (update: <{ _id.id: "LORA-N-0", _id.type: "LoraDevice", _id.servicePath: "/lorattn" }, { $set: { attrs.analog_in_1: { value: 96.64, type: "Number", md: { TimeInstant: { type: "DateTime", value: 1593079623.0 } }, mdNames: [ "TimeInstant" ], creDate: 1593079328, modDate: 1593079623 }, attrs.TimeInstant: { value: 1593079623.0, type: "DateTime", mdNames: [], creDate: 1593079328, modDate: 1593079623 }, modDate: 1593079623, lastCorrelator: "9e10fc36-b6cb-11ea-9471-0242ac1a0005" }, $unset: { location: 1, expDate: 1 } }>)
fiware-orion   | time=Thursday 25 Jun 10:07:03 2020.480Z | lvl=INFO | corr=9e10fc36-b6cb-11ea-9471-0242ac1a0005 | trans=1593079220-399-00000000015 | from=172.26.0.6 | srv=atosioe | subsrv=/lorattn | comp=Orion | op=logMsg.h[1844]:lmTransactionStart | msg=Starting transaction to http://quantumleap:8668/v2/notify
fiware-orion   | time=Thursday 25 Jun 10:07:03 2020.480Z | lvl=INFO | corr=9e10fc36-b6cb-11ea-9471-0242ac1a0005 | trans=1593079220-399-00000000014 | from=172.26.0.6 | srv=atosioe | subsrv=/lorattn | comp=Orion | op=logMsg.h[1874]:lmTransactionEnd | msg=Transaction ended
fiware-orion   | time=Thursday 25 Jun 10:07:03 2020.481Z | lvl=INFO | corr=9e10fc36-b6cb-11ea-9471-0242ac1a0005 | trans=1593079220-399-00000000015 | from=172.26.0.6 | srv=atosioe | subsrv=/lorattn | comp=Orion | op=httpRequestSend.cpp[550]:httpRequestSendWithCurl | msg=Sending message 5 to HTTP server: sending message of 586 bytes to HTTP server
fiware-iot-agent | time=2020-06-25T10:07:03.481Z | lvl=DEBUG | corr=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | trans=c80c1c8b-d05c-4ad5-b865-0ae7b58aa319 | op=IoTAgentNGSI.NGSIService | srv=atosioe | subsrv=/lorattn | msg=Received the following response from the CB: Value updated successfully
fiware-iot-agent |  | comp=IoTAgent
fiware-iot-agent | {"timestamp":"2020-06-25T10:07:03.481Z","level":"info","message":"Observations sent to CB successfully for device "}
quantumleap_1  | INFO:reporter.reporter:Received payload: [{'id': 'LORA-N-0', 'type': 'LoraDevice', 'analog_in_1': {'type': 'Number', 'value': 96.64, 'metadata': {'dateCreated': {'type': 'DateTime', 'value': '2020-06-25T10:02:08.00Z'}, 'dateModified': {'type': 'DateTime', 'value': '2020-06-25T10:07:03.00Z'}}}}]
quantumleap_1  | INFO:geocoding.geocoding:Cannot add location to entity (type: "LoraDevice", id: "LORA-N-0"), missing "address" attribute.
quantumleap_1  | INFO:translators.factory:Env variable QL_CONFIG not set, using default value of: 
quantumleap_1  | INFO:translators.factory:no config file specified, using defaults.
quantumleap_1  | INFO:translators.factory:Backend selected for tenant 'atosioe' is: crate
quantumleap_1  | INFO:reporter.reporter:Notification successfully processed
quantumleap_1  | 172.26.0.5 - - [25/Jun/2020 10:07:03] "POST /v2/notify HTTP/1.1" 200 -
quantumleap_1  | INFO:werkzeug:172.26.0.5 - - [25/Jun/2020 10:07:03] "POST /v2/notify HTTP/1.1" 200 -
fiware-orion   | time=Thursday 25 Jun 10:07:03 2020.605Z | lvl=INFO | corr=9e10fc36-b6cb-11ea-9471-0242ac1a0005 | trans=1593079220-399-00000000015 | from=172.26.0.6 | srv=atosioe | subsrv=/lorattn | comp=Orion | op=httpRequestSend.cpp[570]:httpRequestSendWithCurl | msg=Notification Successfully Sent to http://quantumleap:8668/v2/notify
fiware-orion   | time=Thursday 25 Jun 10:07:03 2020.606Z | lvl=INFO | corr=9e10fc36-b6cb-11ea-9471-0242ac1a0005 | trans=1593079220-399-00000000015 | from=172.26.0.6 | srv=atosioe | subsrv=/lorattn | comp=Orion | op=httpRequestSend.cpp[579]:httpRequestSendWithCurl | msg=Notification response OK, http code: 200
fiware-orion   | time=Thursday 25 Jun 10:07:03 2020.606Z | lvl=INFO | corr=9e10fc36-b6cb-11ea-9471-0242ac1a0005 | trans=1593079220-399-00000000015 | from=172.26.0.6 | srv=atosioe | subsrv=/lorattn | comp=Orion | op=logMsg.h[1874]:lmTransactionEnd | msg=Transaction ended

And here in Crate I can also see details of my device like analog_in_1

Captura de pantalla_2020-06-25_12-14-57

However, ... I was reading the docs about grafana and I was wondering ... Should I use crate:4200 for the host when I configure the data source in Grafana? Because I think it should be localhost:4200 :thinking: I guess I could TIAS

Thank you so much again and I'm closing this because the initial problem is apparently solved :wink:

jfernandz commented 4 years ago

@c0c0n3 Do you recommend me to use your docker-compose.yml?

Now Grafana cannot reach CrateDB :thinking:

Captura de pantalla_2020-06-25_12-33-56

c0c0n3 commented 4 years ago

Do you recommend me to use your docker-compose.yml

That's just a test file but I suppose you could use it as a starting point :-)

Should I use crate:4200 for the host when I configure the data source in Grafana? Now Grafana cannot reach CrateDB

Yes, if you're using a Docker compose network, then the Crate container should be reachable using the crate host---as long as crate is also the service name in your Docker compose, if my memory serves me well. Did you try using crate:4200 when setting up the Grafana data source, like you suggested? It should work as far as I can tell since your Crate service exposes port 4200?

jfernandz commented 4 years ago

Thank you so much, @c0c0n3 it's all working apparently but ... I'd like if you don't mind to ask you a few more things,

Thank you again! :smile:

c0c0n3 commented 4 years ago

Why apparently CrateDB exposes the port 5432 despite I didn't indicated such thing in the docker-compose.yml?

Um, not sure. Probably because of this?

https://github.com/crate/docker-crate/blob/master/Dockerfile#L60

(Latest Docker file, haven't checked v3.3.5 but I bet you it's the same :-)

need the redis database and what for? Is it QuantumLeap?

Yes, that's correct. QuantumLeap uses Redis to cache geo-location data. On receiving a notification QL tries to pin down the geo-coords if they're not explicitly specified in the NGSI entity payload. To do that, QL makes a rather expensive call to OpenStreetMap. To reduce latency then, OSM's response gets cached in Redis so that future notifications of the same entity won't have to pay the price twice.

It is not theoretically using CrateDB?

No, while QL could use a CrateDB table as a cache, we found that Redis is much faster both when adding and retrieving entries from the cache.

Thank you again!

Pleasure! :-)