Rello / analytics

Analytics - Open source data warehouse and reporting for Nextcloud
GNU Affero General Public License v3.0
116 stars 32 forks source link

wrong decimal handling for certain numbers #397

Open MMM-Tech-Support opened 3 months ago

MMM-Tech-Support commented 3 months ago

Hey,

I use the JSON Datasource to get the atmospheric pressure via a HTTP GET command. Screenshot 2024-07-08 at 14-00-43 Analytics - Nextcloud It work's quite well.

When I analysed the data I recognized, that everytime the pressure has more than 2 decimal places the value is 1000 times to high.

So the value 1014.375 becomes 1014375 Please see the 3. column.

Screenshot 2024-07-08 at 14-03-11 Barometrischer Luftdruck in hPa - Analytics - Nextcloud

Here is the Graph and the the jumps between 1000 and 1000000 are clearly visible.

Screenshot 2024-07-08 at 14-03-35 Barometrischer Luftdruck in hPa - Analytics - Nextcloud

I checked the JSON file and the values are all correct. So I think there is a bug in the JSON Parser.

Best regards and thanks for your work! Heinrich

MMM-Tech-Support commented 3 months ago

Hey again!

Here are some more informations to isolate the bug. When I run the JSON Datasource in Test Run Mode, i get a correct reading and the value is correct parsed with 3 decimal places.

Screenshot 2024-07-22 at 22-53-33 Analytics - Nextcloud

When I load the same data in non test mode the bug happens and 1014.375 becomes 1014375.

Screenshot 2024-07-22 at 23-07-43 Luftdruck aktuell - Analytics - Nextcloud

Best regards!

Rello commented 3 months ago

Hi, for simpler debugging, can you attach the json example here?

MMM-Tech-Support commented 3 months ago

Hey,

Sure, it's attached. The JSON path to atmospheric pressure is data/values/0 to 720/110_133_828

The first datapoint with more than 2 decimal places is at data/values/1/110_133_828

atmos.json

Please let me know if you need more information and again thanks for your great work!!! Best regards

Rello commented 3 months ago

working for me. I will do some more testing. is the error coming up when loading via the "load now" or during scheduled loads?

Bildschirmfoto 2024-07-29 um 20 48 23

Bildschirmfoto 2024-07-29 um 20 48 42

Rello commented 3 months ago

I found it. single value update is ok. batch update is causing an issue on some values.

by the way: I do not know how you loaded the numbers above. but with the array-type loading, you can get the timestamp also from the json file

data/values{110_133_828 ,read_at_group,110_133_828}

then you have all values from the file. not the perfect format. this can be adjusted later, but I think this might be more what you need?

Rello commented 3 months ago

fyi

Bildschirmfoto 2024-07-30 um 10 46 08

path string:

data/values{Luftdruck,read_at_group,110_133_828}

I can see that you are reporting on your environmental sensors. If you want to discuss a better integration, feel free to get in direct contact with me analytics(at)scherello.de

tatrapikao commented 3 months ago

same when using data from tables app. Value with 3 decimal values is parsed like: 9,600 -> 9,6 but 9,612 -> 9.612