search

jortizcs / Pangia

Anomaly detection in buildings
greenpangia.com
Other
0 stars 0 forks source link

getalarms.js requests wrong timestamps #11

Closed romain-fontugne closed 11 years ago

romain-fontugne commented 11 years ago

I still have difficulties in plotting the graphs. The plots are empty. It seems that getalarms.js fetchs data in TSDB with wrong timestamp. Here is what's I have done: Using chart.php I've tried to plot the following alarm that happened in 2011. 

id=53   username=root start=2011-05-24 09:01:37 end=2011-05-24 19:01:37 label01=CoryHall_5DPB_elt-D_4F_Elevator_.dat label02=CoryHall_gpe-a_elt-A_1F_Misc_.dat deviation=29.077038755

but this is what TsdbShim output:

Apr 03, 2013 9:18:44 AM TsdbShim$AsyncTask run
INFO: query_string=start=2016%2F12%2F23-15%3A46%3A10&m=sum%3Asbs.root.53%7Blabel%3DCoryHall_5DPB_elt-D_4F_Elevator_.dat%7D&end=2007%2F09%2F02-19%3A31%3A55
Apr 03, 2013 9:18:44 AM TsdbShim runTsdbQuery
INFO: start=[2016/12/23-15:46:10,1482475570]
Apr 03, 2013 9:18:44 AM TsdbShim runTsdbQuery
INFO: end=[2007/09/02-19:31:55,1188729115]
Apr 03, 2013 9:18:44 AM TsdbShim runTsdbQuery
INFO: opentsdb url=http://localhost:4242/q?start=2016/12/23-15:46:10&m=sum:sbs.root.53{label=CoryHall_5DPB_elt-D_4F_Elevator_.dat}&end=2007/09/02-19:31:55&ascii
java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:4242/q?start=2016/12/23-15:46:10&m=sum:sbs.root.53{label=CoryHall_5DPB_elt-D_4F_Elevator_.dat}&end=2007/09/02-19:31:55&ascii
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
    at TsdbShim.runTsdbQuery(TsdbShim.java:125)
    at TsdbShim$AsyncTask.run(TsdbShim.java:84)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

The timestamps are completely random...

jortizcs commented 11 years ago

It may be that they're not being interpreted correctly. This took a long time to fix on the php version of getalarms. It's worth going back to that build and looking at that.

romain-fontugne commented 11 years ago

OK, thanks for the pointers. I will look at that today.

romain-fontugne commented 11 years ago

Well... the source of the problem was from a MySQL attribute (on_update CURRENT_TIMESTAMP) that changed start on any update.... I've removed this attribute from the joyment server and I advice you do same on your local deployments

But I now get another problem, the connection between TsdbShim and getalarms.js seems to stop if there is too much data. Here is the error I get: 

undefined:1
99761581421],[1306315006,0.8999999761581421],[1306315106,0.8999999761581421],[
                                                                          ^
SyntaxError: Unexpected end of input
    at Object.parse (native)
    at IncomingMessage.<anonymous> (/home/pangia/repo/Pangia/routes/getalarms.js:145:14)
    at IncomingMessage.EventEmitter.emit (events.js:95:17)
    at IncomingMessage.<anonymous> (_stream_readable.js:699:14)
    at IncomingMessage.EventEmitter.emit (events.js:92:17)
    at emitReadable_ (_stream_readable.js:382:10)
    at emitReadable (_stream_readable.js:377:5)
    at readableAddChunk (_stream_readable.js:142:7)
    at IncomingMessage.Readable.push (_stream_readable.js:112:10)
    at HTTPParser.parserOnBody [as onBody] (http.js:140:22)
make: *** [server] Error 8
romain-fontugne commented 11 years ago

It is fixed! The HTTP response is sent in several chunks if there is too may data. This is now handled in getalarm.js

jortizcs commented 11 years ago

@romain-fontugne Oh, interesting, but how was that related to the timestamps?

romain-fontugne commented 11 years ago

To test the graph I had to manually modify the alarms through phpmyadmin and every time I modified an alarm its "start" value was set to "NOW()" (because of an option that was selected in MySQL)..... I didn't see that at first because phpmyadmin do not refresh the column automatically updated when the modification are done inline... (so it actually an issue with phpmyadmin! :-( )

Anyway, I advice you to remove the "on_update CURRENT_TIMESTAMP" attribute for the start column as it raises undesired values. I also cleaned a bit the code in getAlarms that manipulates the alarms timestamps.