Intermittent Wrong Timestamps

[ahaas]

The previous issue #37 seems to be fixed when I first make the database, however when I come back to my database a day later, all the timestamps are 1000x again. Clearing my /tmp files temporarily resolves it. (This has happened to me several times.)

[gtfierro]

Have you tried directing the databases to somewhere other than /tmp? Have you tried generating timestamps manually and inserting into sMAP outside of a driver? When you say 1000x, do you mean that timestamps are smaller than what you inserted or larger than what you inserted? You should read the documentation here (https://gtfierro.github.io/giles/interface/), which mentions that timestamps are returned in milliseconds by default.

[ahaas]

Ok, I'll take a look. Actually looking at the results more closely, it does seem to be the case that Giles is returning all timestamps in milliseconds, including the ones meant to be in milliseconds. However, retrieving the UnitofTime tells me that most of the streams are in units of seconds. So the timestamps for many streams are inconsistent with the UnitofTime, and this happens when I come back to my /tmp database some time later after the data is first inputted by the fake drivers.

~/giles_test❯ curl -X POST -d "select * where uuid='c76e3e87-65f0-5d4d-8e1a-22fc5684f5a0';" http://localhost:8079/api/query | jprint
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   730  100   671  100    59  90030   7916 --:--:-- --:--:-- --:--:-- 95857
[
  {
    "Actuator": {
      "path": "/tasklighting/group1/sat_act", 
      "uuid": "4662d038-bc2f-5122-a348-846f8b23f35d"
    }, 
    "Metadata": {
      "Building": "Soda Hall", 
      "Device": "Lighting Controller", 
      "DeviceID": "905d2818-aa82-11e4-acab-0cc47a0f7eea", 
      "Driver": "smap.drivers.lights.virtuallight", 
      "Group": "1", 
      "LightingZone": "410 Soda", 
      "Model": "Virtual Light", 
      "Role": "Task Lighting", 
      "Site": "0273d18f-1c03-11e4-a490-6003089ed1d0", 
      "SourceName": "Demo Driver", 
      "System": "Lighting", 
      "Type": "Command", 
      "configured": "True"
    }, 
    "Path": "/tasklighting/group1/sat", 
    "Properties": {
      "ReadingType": "long", 
      "Timezone": "America/Los_Angeles", 
      "UnitofMeasure": "Saturation", 
      "UnitofTime": "s"
    }, 
    "uuid": "c76e3e87-65f0-5d4d-8e1a-22fc5684f5a0"
  }
]
~/giles_test❯ curl -X POST -d "select data before now limit 5 where uuid='c76e3e87-65f0-5d4d-8e1a-22fc5684f5a0';" http://localhost:8079/api/query | jprint  
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   247  100   166  100    81  10312   5031 --:--:-- --:--:-- --:--:-- 11066
[
  {
    "Readings": [
      [
        1430237612000.0, 
        80
      ], 
      [
        1430237611000.0, 
        80
      ], 
      [
        1430237610000.0, 
        80
      ], 
      [
        1430237609000.0, 
        80
      ], 
      [
        1430237608000.0, 
        80
      ]
    ], 
    "uuid": "c76e3e87-65f0-5d4d-8e1a-22fc5684f5a0"
  }
]

I can try this with a different database location and other data.

[gtfierro]

Yes, this is the expected behavior. The plotting tools and old sMAP tools require milliseconds. Further work on the query language will allow the return of data in other units, but for now we don't have any streams that require finer resolution.

[ahaas]

From your documentation, it seems that the timestamp should be consistent with UnitofTime?

"Readings": [                       // This is an array of (timestamp, value) tuples. Timestamps should be
            [                               //  consistent with Properties/UnitofTime
                1351043674000,              // A timestamp
                0                           // A numeric value
            ],

Secondly, my finding is that all streams return in milliseconds only after the database is old, for some reason. If I make a new database and start putting data in it, you can see that the timestamps now match the UnitofTime again, and are in seconds not milliseconds. So sometimes, it the timestamps are in seconds, and sometimes they are in milliseconds, but the UnitofTime has always been seconds for this stream.

~/giles_test❯ curl -X POST -d "select * where uuid='c76e3e87-65f0-5d4d-8e1a-22fc5684f5a0';" http://localhost:8079/api/query | jprint
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   730  100   671  100    59  94948   8348 --:--:-- --:--:-- --:--:--  109k
[
  {
    "Actuator": {
      "path": "/tasklighting/group1/sat_act", 
      "uuid": "4662d038-bc2f-5122-a348-846f8b23f35d"
    }, 
    "Metadata": {
      "Building": "Soda Hall", 
      "Device": "Lighting Controller", 
      "DeviceID": "905d2818-aa82-11e4-acab-0cc47a0f7eea", 
      "Driver": "smap.drivers.lights.virtuallight", 
      "Group": "1", 
      "LightingZone": "410 Soda", 
      "Model": "Virtual Light", 
      "Role": "Task Lighting", 
      "Site": "0273d18f-1c03-11e4-a490-6003089ed1d0", 
      "SourceName": "Demo Driver", 
      "System": "Lighting", 
      "Type": "Command", 
      "configured": "True"
    }, 
    "Path": "/tasklighting/group1/sat", 
    "Properties": {
      "ReadingType": "long", 
      "Timezone": "America/Los_Angeles", 
      "UnitofMeasure": "Saturation", 
      "UnitofTime": "s"
    }, 
    "uuid": "c76e3e87-65f0-5d4d-8e1a-22fc5684f5a0"
  }
]
~/giles_test❯ curl -X POST -d "select data before now limit 5 where uuid='c76e3e87-65f0-5d4d-8e1a-22fc5684f5a0';" http://localhost:8079/api/query | jprint  
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   248  100   167  100    81    187     90 --:--:-- --:--:-- --:--:--   187
[
  {
    "Readings": [
      [
        1430239439.0, 
        80
      ], 
      [
        1430239438.0, 
        80
      ], 
      [
        1430239437.0, 
        80
      ], 
      [
        1430239436.0, 
        80
      ], 
      [
        1430239435.0, 
        80
      ]
    ], 
    "uuid": "c76e3e87-65f0-5d4d-8e1a-22fc5684f5a0"
  }
]

[gtfierro]

Documentation is for timestamp input, not output.

Does "database" mean readingdb, mongo or both/

[gtfierro]

Also, have you tried manually inserting data?

[ahaas]

not yet, I will try that soon

On Tue, Apr 28, 2015 at 9:49 AM, Gabe Fierro notifications@github.com wrote:

Also, have you tried manually inserting data?

Reply to this email directly or view it on GitHub: https://github.com/gtfierro/giles/issues/40#issuecomment-97134772