Float variables doesn't push data to DB

[mapedraza]

I am having troubles with data persistence using that Enabler. After checking the FIWARE Tutorial, I modify the entities and so on and I am not be able to view the data in grafana (also appears on crate DB as a null), but all data seems good in Context Broker. I also check the notification content and It works fine. Also if I use the examples in the tutorial it works fine.

For that reason i fake the system sending a notification manually to the notification endpoint to debug the problem

Sending the same request that context broker sends (float data as Context Brokers send itself) i obtain a problem. The request was:

{
  "subscriptionId": "5c12f90a2adc243203c276a2",
  "data": [
    {
      "id": "Device:001",
      "type": "Device",
      "ambient_temperature": {
        "type": "Number",
        "value": "4.4",
        "metadata": {
          "TimeInstant": {
            "type": "ISO8601",
            "value": "2018-12-14T16:24:25.429Z"
          }
        }
      }
    }
  ]
}

I obtained that response:

{
    "detail": "The server encountered an internal error and was unable to complete your request.  Either the server is overloaded or there is an error in the application.",
    "status": 500,
    "title": "Internal Server Error",
    "type": "about:blank"
}

I just check the number value and it is a string ("value": "4.4")

I just changed the value to a real JSON as following

{
  "subscriptionId": "5c12f90a2adc243203c276a2",
  "data": [
    {
      "id": "Device:001",
      "type": "Device",
      "ambient_temperature": {
        "type": "Number",
        "value": 4.4,
        "metadata": {
          "TimeInstant": {
            "type": "ISO8601",
            "value": "2018-12-14T16:24:25.429Z"
          }
        }
      }
    }
  ]
}

And i obtained the following response: "Notification successfully processed"

Also reviewing the Grafana front end, the paremter apear as a integer (value for)

Checking the crate DB I obtained the same value, 4, and not 4.4

I think the notification endpoint must accept the same syntax that context broker offers, but also making a "Custom notification" with the same syntax but putting the value as a integer JSON will not work properly, because the system will not save the decimal part of the number, treating it as an integer.

[chicco785]

To take advantage of the performances of crate, the data type of crate columns is defined the first time an attribute is pushed to crate. Probably, the issue is cause by the fact that you were using a wrong type in the JSON. Did you try dropping the table and using the correct JSON?

Cheers, Federico

Il giorno 16 dic 2018, alle ore 16:02, mapedraza notifications@github.com ha scritto:

I am having troubles with data persistence using that Enabler. After checking the FIWARE Tutorial, I modify the entities and so on and I am not be able to view the data in grafana (also appears on crate DB as a null), but all data seems good in Context Broker. I also check the notification content and It works fine. Also if I use the examples in the tutorial it works fine.

For that reason i fake the system sending a notification manually to the notification endpoint to debug the problem

Sending the same request that context broker sends (float data as Context Brokers send itself) i obtain a problem. The request was:

{ "subscriptionId": "5c12f90a2adc243203c276a2", "data": [ { "id": "Device:001", "type": "Device", "ambient_temperature": { "type": "Number", "value": "4.4", "metadata": { "TimeInstant": { "type": "ISO8601", "value": "2018-12-14T16:24:25.429Z" } } } } ] }

I obtained that response:

{ "detail": "The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.", "status": 500, "title": "Internal Server Error", "type": "about:blank" }

I just check the number value and it is a string ("value": "4.4")

I just changed the value to a real JSON as following

{ "subscriptionId": "5c12f90a2adc243203c276a2", "data": [ { "id": "Device:001", "type": "Device", "ambient_temperature": { "type": "Number", "value": 4.4, "metadata": { "TimeInstant": { "type": "ISO8601", "value": "2018-12-14T16:24:25.429Z" } } } } ] }

And i obtained the following response: "Notification successfully processed"

Also reviewing the Grafana front end, the paremter apear as a integer (value for)

Checking the crate DB I obtained the same value, 4, and not 4.4

I think the notification endpoint must accept the same syntax that context broker offers, but also making a "Custom notification" with the same syntax but putting the value as a integer JSON will not work properly, because the system will not save the decimal part of the number, treating it as an integer.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[chicco785]

@mapedraza did you check my suggestion?

[mapedraza]

Yes, after dropping the table it works well. I provisioned the device again as a "Number" and the Context Broker send the notifications properly. I think could be nice to implement into data-sink to handle Number (float) data as a strings

Thanks for your support! Miguel A.

[chicco785]

I am not sure what you are proposing.

storing data as string in crate, would not make them querable. what may be possible is, if context brokers says that type is Number, and anyhow sends a string, QL can cast the value to float. is that your idea?

On 21 Dec 2018, at 10:52, mapedraza notifications@github.com wrote:

Yes, after dropping the table it works well. I provisioned the device again as a "Number" and the Context Broker send the notifications properly. I think could be nice to implement into data-sink to handle Number (float) data as a strings

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/smartsdk/ngsi-timeseries-api/issues/123#issuecomment-449337724, or mute the thread https://github.com/notifications/unsubscribe-auth/AAvcAkrRtRxxs-9BW-a8FOmvX8Ee4VHhks5u7K9EgaJpZM4ZVRsk.

[chicco785]

@c0c0n3 if @mapedraza confirms, can you check you open a issue for the feature? so we then can close this thread

[mapedraza]

@chicco785 Yes, that is the idea! If the data type received in the notification is Number, the API that manage the received data can cast from string to float if the JSON contains the data as a string, avoiding to fail as I described above.