SensorKit timestamp inconsistency.

carlan1 commented 1 year ago

Some of the time and date intervals report either in seconds, in the case of time intervals, or seconds since January 1 2001, as is the case for date intervals. We will need this data to be reported in ms or unix timestamps since 1970.

I post an example below, retrieved from the visits feature, where the arrival and departure intervals are reported as seconds since 2001: {'distanceFromHome': 3321.593688888473, 'arrivalDateInterval': {'start': 686148300, 'duration': 900}, 'locationCategory': 2, 'departureDateInterval': {'start': 686178000, 'duration': 900}}

Another example below highlights a time interval reported in seconds, from the messages use feature. The duration key appears to be in seconds.

{'timestamp': 1664672399998,
  'sensor': 'com.apple.sensorkit.messages_usage',
  'data': {'duration': 1800,
   'totalOutgoingMessages': 2,
   'totalIncomingMessages': 4,
   'totalUniqueContacts': 4}},

ZCOEngineer commented 1 year ago

@carlan1 we shall check this and get back

jijopulikkottil commented 1 year ago

The "timestamp" key, which is added manually. This is the time interval in ms since 1970 [and keep same in both iOS and Android].

The data is the content which is getting as a fetch result through Apple API. For the SensorKit data, we are just sending the received data as it is by adapting default encoding.

So the default Swift implementation of encoding works when we send data to server. The default Date encoding of Swift is 'seconds from ReferenceDate' which is seconds from Jan 2001.

So to fix this issue, we have to implement our own model and encoding for all SensorKit data. Let us know do we need to proceed this way.

jijopulikkottil commented 1 year ago

@carlan1 edited above comment!

carlan1 commented 1 year ago

Yes, we would like to implement this model for our SensorKit sensors as we have for the other LAMP sensors to keep them consistent with each other.

carlan1 commented 1 year ago

@jijopulikkottil When you create the new model, will the SensorSpec identifier names be in the usual "lamp.x" format, or will they remain as "com.apple.sensorkit.x"?

jijopulikkottil commented 1 year ago

Spec identifier names will remain the same.

jijopulikkottil commented 1 year ago

Hi @carlan1 do we need to add text representation of integer (number) value ? e.g placement in AmbientLightSensor. its value is a number

we have done the similar for sleep data. can see value and it representation

{
"timestamp": 1618240969253,
"sensor": "lamp.sleep",
"data": {
"value": 0
"representation": "in_bed"
}
}

carlan1 commented 1 year ago

Because there are 9 possibilities for placement, this would make the schema very long. We don't need string values for now, as long as the integer values line up with the ones listed in https://developer.apple.com/documentation/sensorkit/srambientlightsample/sensorplacement

michaelmenon commented 1 year ago

@carlan1 we will be logging to see if we get duplicate SRVisit data for different timestamps within few seconds apart. We will check on this and we can discuss on this in our next weeks call.

carlan1 commented 1 year ago

Ok, so here there are two issues:

Duplicate events with identical timestamps. We want these removed and for the fix to be pushed as soon as it's ready.
Events with different timestamps but with timestamps only a few seconds apart. For now, test to see if this is a problem. If so, we can discuss what to do about it on our call.

jijopulikkottil commented 1 year ago

We have handled Date properties to reported in ms since 1970. So all date fields treated as same. [attached some SRVisit data below]
Because there are 9 possibilities for placement, this would make the schema very long. We don't need string values for now, as long as the integer values line up with the ones listed in https://developer.apple.com/documentation/sensorkit/srambientlightsample/sensorplacement

Not really, we will add only the text representation of that integer value. We can see it in the sample SRVisit data attached below.

"locationCategory": 0, 
"locationCategoryRepresentation": unknown

Also the order is not same as which is listed in that page.

Please confirm if we don't want the "...Representation" field.

Duplicate events with identical timestamps. We want these removed and for the fix to be pushed as soon as it's ready.

This issue will be handled with #699

4.

Events with different timestamps but with timestamps only a few seconds apart. For now, test to see if this is a problem. If so, we can discuss what to do about it on our call.

Duplicate events with almost same timestamp will be handled with 699.

4.1. If we see the sample SRVisit data attached here, there is multiple events generated in almost same time (1666249084xxx). All timestamps difference in few milliseconds only.

All events data are different here. So this case can happen.

 ["sensor": "com.apple.sensorkit.visits", "timestamp": 1666249084758, "data": [ "departureDateInterval": {
    duration = 900;
    end = 1666150200000;
    start = 1666149300000;
}"arrivalDateInterval": {
    duration = 900;
    end = 1666098000000;
    start = 1666097100000;
},"locationCategoryRepresentation": unknown, ”identifier": 87651401-7D3A-461D-98E8-C330AE3DA067, "locationCategory": 0, , "distanceFromHome": 3033.690786950916,] ]
["sensor": "com.apple.sensorkit.visits", "timestamp": 1666249084769, "data": ["departureDateInterval": {
    duration = 900;
    end = 1666151100000;
    start = 1666150200000;
}, "arrivalDateInterval": {
    duration = 900;
    end = 1666151100000;
    start = 1666150200000;
}, "locationCategoryRepresentation": unknown, "identifier": 87651401-7D3A-461D-98E8-C330AE3DA067, "locationCategory": 0, "distanceFromHome": 3033.690786950916]]
["sensor": "com.apple.sensorkit.visits", "timestamp": 1666249084776, "data": ["departureDateInterval": {
    duration = 900;
    end = 1666162800000;
    start = 1666161900000;
}, "arrivalDateInterval": {
    duration = 900;
    end = 1666154700000;
    start = 1666153800000;
}, "locationCategoryRepresentation": unknown, "identifier": 87651401-7D3A-461D-98E8-C330AE3DA067, "locationCategory": 0, "distanceFromHome": 3033.690786950916]]
["sensor": "com.apple.sensorkit.visits", "timestamp": 1666249084783, "data": ["departureDateInterval": {
    duration = 900;
    end = 1666239300000;
    start = 1666238400000;
}, "arrivalDateInterval": {
    duration = 900;
    end = 1666165500000;
    start = 1666164600000;
}, "locationCategoryRepresentation": unknown, "identifier": 87651401-7D3A-461D-98E8-C330AE3DA067, "locationCategory": 0, "distanceFromHome": 3033.690786950916]]
["sensor": "com.apple.sensorkit.visits", "timestamp": 1666249084788, "data": ["departureDateInterval": {
    duration = 900;
    end = 1666153800000;
    start = 1666152900000;
}, "arrivalDateInterval": {
    duration = 900;
    end = 1666152900000;
    start = 1666152000000;
}, "locationCategoryRepresentation": unknown, "identifier": 84B34B66-8150-4371-8675-35180A2FD049, "locationCategory": 0, "distanceFromHome": 1003.829408192863]]
["sensor": "com.apple.sensorkit.visits", "timestamp": 1666249084792, "data": ["departureDateInterval": {
    duration = 900;
    end = 1666163700000;
    start = 1666162800000;
}, "arrivalDateInterval": {
    duration = 900;
    end = 1666163700000;
    start = 1666162800000;
}, "locationCategoryRepresentation": unknown, "identifier": 84B34B66-8150-4371-8675-35180A2FD049, "locationCategory": 0, "distanceFromHome": 1003.829408192863]]
["sensor": "com.apple.sensorkit.visits", "timestamp": 1666249084798, "data": ["departureDateInterval": {
    duration = 900;
    end = 1666152000000;
    start = 1666151100000;
}, "arrivalDateInterval": {
    duration = 900;
    end = 1666152000000;
    start = 1666151100000;
}, "locationCategoryRepresentation": unknown, "identifier": 96DA9B90-2710-4F45-B093-DEF5E1AC9A0B, "locationCategory": 0, "distanceFromHome": 545.8496617101068]]

carlan1 commented 1 year ago

Good, this is what we want. We would also like 'duration' in ms if it's not already.
If the order is not the same as the page, and you are only adding a single string representation for the specific placement of the data point, then in fact we would prefer to include 'representation'
Ok, please comment on issue #699 that the fix for that issue will be a fix for all of the new sensors
Yes, I see that these events are only a few ms apart. We can discuss how to deal with those events on Wednesday

jijopulikkottil commented 1 year ago

duration property is NOT handled in ms. There are more duration property in other sensors like audio frameDuarion, pauseDuration and speechDuration in SpeechSensorKit data. In DeviceUsageReport, there are duration, unlockDuration, inpoutsessionDuration. Also duration in KeyboradMetrics, MessageUsage, PhoneUsage, SRVisit. All can convert to milliseconds. ok.
Ok. Following is an array of sessionType in Device-Usage data. So we are adding its representations. hope this is fine.

Done.

carlan1 commented 1 year ago

Yes, everything should be in ms. Everything currently in s should be converted to ms.

Yes, we would like these new properties added for session type.

jijopulikkottil commented 1 year ago

It is fixed. The staging pointed version 2022.10.26 has been submitted last week

carlan1 commented 1 year ago

Appears fixed in staging

carlan1 commented 1 year ago

@jijopulikkottil For the device usage sensor, are the usageTime variables given in ms, or s?

jijopulikkottil commented 1 year ago

@carlan1 Its in s. We fixed all 'duration' fields. But this field usageTime (TimeInterval type) is not converting to ms. We can convert this too. Also found one more field 'speechStartTimestamp' in SpeechRecognition which has TimeInterval type (In swift TImeInterval type fields are in Seconds). We can fix all.

carlan1 commented 1 year ago

Yes we would like everything in ms

jijopulikkottil commented 1 year ago

Completed and the code has been pushed. Staging pointed test flight build 2023.2.23 (403) is available now. [QA testing: waiting for the data to verify. not received enough data from device]

jijopulikkottil commented 1 year ago

QA testing is completed.

ertjlane commented 1 year ago

This appears to be fixed.

BIDMCDigitalPsychiatry / LAMP-platform

SensorKit timestamp inconsistency. #701