Accessing accelerometer and other sensor data programmatically?

[vr00n]

Hey there - How do I (programmatically) access the accelerometer and other sensor metadata for an individual image?

[james2432]

I think they are stored in the zip/tar file (seems to be sequential file) that is included with the photos on the phones storage.

[bogdans-telenav]

Currently there is no API for this and as far as there is no plan to make one, but as @james2432 sad you can access any sensor metadata from the file stored with the photos on the phone storage. The file has all the sensors written there. The sensors are logged from several threads so you need to sort the file first by timestamp and after that just take the latest informations about a certain image.

[vr00n]

Thanks so much for the quick response. This is mighty helpful. If there is a provision to perhaps stream the sensor data to some cloud endpoint (S3 / FTP / Google Sheets) - would unlock many many potential applications. Is there a way I can nominate this as a feature?

[alephcero]

I accessed the data in from file within track.txt.gz used by OSC to store sensor data This is the phone info from the header of the file : motorola Moto G (4);7.0;1.1.6;2.0.4;video

I got 21 columns. From those, clearly the first one is the timestamp, the next 2 are gps data with long,lat. Here is a jupyter notebook with the code to read the file. https://github.com/alephcero/OSCstats/blob/master/loadOSCdata.ipynb

@bogdans-telenav would you happen to know which columns store the accelerometer data?

[bogdans-telenav]

@alephcero this is a row example for version 1.1.6 timestamp;lon;lat;elevation;horizontal_accu;GPSspeed;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compas;vIndex;tFIndex;gravityX;gravityY;gravityZ;OBD2Speed;vertical_accu

[bogdans-telenav]

@vr00n if you check the API for a sequence you will get a link to the metadata file that is stored on OSC servers.

[alephcero]

Based on: https://github.com/openstreetcam/openstreetview.org/issues/109

I used those variable names and kept gravity and acelerometer

[vr00n]

Thanks @bogdans-telenav - I emailed hello@openstreetcam.org for directions to access the API.

http://beta.openstreetcam.org/1.0/sequence/export/167913/ did not work.

Thanks

[bogdans-telenav]

@vr00n what do you want to do? I can probably help you here.

[bogdans-telenav]

@vr00n Just in case here is a example on how to get a sensor link to the metadata file.

• step 1: call - http://openstreetcam.org/details (this is called by the website when you preview a track) with the appropriate parameters
• step 2: in the response find { .... "osv" : { ...... "meta_data_filename": "some_text_here" ..... } .....}
• step 3: compose the link: openstreetcam.org/"some_text_here"

[vr00n]

THANKS @bogdans-telenav! I tested this, it works and I am able to see all my track data from the /details page ...though i'm not sure I fully understand how to correctly construct the URL (post the correct headers) given that I know my track-id

Ideally, for a given track-id would like to construct a URL call that, returns the following:

UnixTimeStamp,Latitude,Longitude,AccelerometerX,AccelerometerY,AccelerometerZ,LinktoFullRes.ImageFile

So:

149507697300,40.745584, -73.993266,0.3,0.4,0.5,http://storage3.openstreetcam.org/files/photo/2017/5/12/lth/167913_e0a45_5915cd5236ffd.jpg

Thank you to you and team Telenav for the support. So cool you are doing this. Reverberating throughout the open mapping community!

[bogdans-telenav]

@vr00n there is no such API and as far as i know there is no plan to do this. The only way for now is to parse the metadata, and figure witch accelerometer data you want to keep. About the details API: http://openstreetcam.org/details - required parameter id = 67183 the result is:
{ osv:{date_added: "Apr 05, 2017", platform: "iPhone6", user: "Gropitza",…} address:"Strada Tipografiei, Someșeni, Cluj-Napoca, Romania, RO" changed:"0" client_total:"22" count_active_photos:"29" date_added:"Apr 05, 2017" is_rotating:false meta_data_filename:"storage3/files/photo/2017/4/5/67183_16905_58e4ce05adc87.txt" obd_info:"0" owner:false photos:[{id: "24945261", sequence_id: "67183", sequence_index: "0", lat: "46.773364", lng: "23.593080",…},…] platform:"iPhone6" recognitions:"0" reviewed:"0" upload_history:{id: "27553", sequence_id: "67183", user_id: "28", user_category: "REGULAR", distance: "0.01",…} user:"Gropitza" status:{apiCode: "600", apiMessage: " The request has been processed without incidents ", httpCode: 200,…} } Now to access the sensors file we construct the link like this: www.openstreetcam.org/ storage3/files/photo/2017/4/5/67183_16905_58e4ce05adc87.txt

[vr00n]

Woohoo! Eureka for us! Thanks @bogdans-telenav .

One thing that seems confusing looking at the meta_data_filename txt file is how the "timestamp" tag is created for each reading.

Looks v confusing. It appears that the timestamp shifts only after 64 or 128 readings. Why?

See here https://docs.google.com/spreadsheets/d/1yyvSBeozeMdbO-swGIQ7pz80arXtpBumywGHOYlwSJ4/edit?usp=sharing that contains a recent track.

[bogdans-telenav]

@vr00n At a glance my answer would be that the precision of your reading is in seconds, if you would use the milliseconds precision you would see that there are differences.

[vr00n]

@bogdans-telenav hmm not sure we are looking at the same thing. Im looking at the details JSON. See here for parameter id = 67183 https://jsonblob.com/75bb1d29-3f50-11e7-ae4c-d50be0b1926d

search for timestamp and you see what I am referring to

[bogdans-telenav]

@vr00n "Looks v confusing. It appears that the timestamp shifts only after 64 or 128 readings. Why?" So if we are talking about the date_added or timestamp in the response, there is a misunderstanding. This timestamps represent the server processing time, date of upload, or something like that (I really don't know the exact meaning) but i am certain that these timestamps are not the same (or at least not as precise) with the ones that are present in the sensors file (the OSC team refers to it as the "metadata file"). So what would be the right solution? Steps:

1. Download the metadata file (as described in this thread)
2. Parse it, get your acceleration and photo entries with there timestamps (as described in this thread)
3. If you would like to have photo-acceleration then keep only the timestamp of the photo on that photo-acceleration aggregation.
4. Add relevant info: photo link, way id, matched_lat, matched_long to your aggregation combining the metadata photo-acceleration with the request made at step 1

Actual fact: The only relevant infos that are not present in the metadata and can not be inferred: photo link, way id, matched_lat, matched_long, sequence_id, id

Heads up: We are working on API v2 so unnecessary info will be removed from API response.

[patwater]

@bogdan-telenav thanks for this very helpful thread! Question on the columns in the metadata file copied below from your message on 5/2/17:

"timestamp;lon;lat;elevation;horizontal_accu;GPSspeed;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compas;vIndex;tFIndex;gravityX;gravityY;gravityZ;OBD2Speed;vertical_accu"

Would it be correct to infer that accelerationX, acclerationY, and accelerationZ are the raw readings from the phone and gravityX, gravityY, and gravityZ are those readings transformed into units of g (i.e. the standard earth gravity measure of acceleration of 9.8 m/ s**2)?

Is there documentation detailing how those fields are derived from the phone's raw data (or could you point to where in the OSC code this is occurring)?

PS see here for a motivating example of how we're using OSC data to help improve street maintenance: https://twitter.com/BSSLosAngeles/status/937436785411366912

See commute.streetsdatacollaborative.org for an example dashboard

CC @vr00n @dmarulli

[patwater]

Hmmmm upon closer inspection seems that the gravity x / y / z values range less than absolute magnitude of 0.6 and the accelerometer x / y / z values range +/- 8 so the hypothesis in the previous message doesn't seem to hold

[vr00n]

@patwater @dmarulli

for Ios see here - https://github.com/openstreetcam/ios/blob/30cfb11a9f0450eec0c7036062643dd29ffbef0d/OpenStreetView/OSVTrackLogger.m

for Android see here - https://github.com/openstreetcam/android/search?p=1&q=acceleration&type=&utf8=%E2%9C%93

A good test would be to use SensorLog https://itunes.apple.com/us/app/sensorlog/id388014573?mt=8 that exposes native sensor readings and conduct 2 data collections.

1 using sensorlog and the other using OSC under similar conditions

compare OSC uploaded track with SensorLog capture to check for variations

[dmarulli]

@bogdans-telenav - thanks for all the help last May.

Along with @patwater and @vr00n, I have been diving into using OSC imagery and sensor metadata to improve street quality. As @patwater alluded to, we are currently able to process OSC data and pipe them to a UI we built. We recorded a track and nudged the Los Angeles Bureau of Street Services with one of our OSC-powered maps. That interaction/map is here (https://twitter.com/BSSLosAngeles/status/937436785411366912).

We have been building out the infrastructure to leverage not only tracks that we actively record, but also existing OSC tracks recorded by the crowd to do this at the scale of a city.

This has been going very well so far, however I am noticing a strange pattern:

for tracks with imagery indexed in the meta_data_file with tFIndex, there seem to be roughly equal counts of photos in the http://openstreetcam.org/details "API" output, and meta_data_file records with non-null tFIndex. This situation allows us to aggregate to a single accelerometer reading per image.

However for tracks with imagery indexed in the meta_data_file with vIndex, there are far fewer meta_data_file records with non-null vIndex than photos in the API output. This situation prevents us from assigning an accelerometer reading to each image.

A concrete example (not the only one):

For track 481059, there are 5980 photos in the API, and 5980 records in the meta_data_file with non-null tFIndex (http://openstreetcam.org/storage5/files/photo/2017/7/10/481059_16905_59630592b4b94.txt)

For track 11091, there are 854 photos in the API json, but only 55 records in the meta_data_file with non-null vIndex (http://storage2.openstreetcam.org/files/photo/2016/8/2/11091_d875a_57a08df0c002c.txt).

Please let me know if I can clarify anything. Thanks again.

[bogdans-telenav]

@dmarulli I am really excited to hear about your project, and that OSC is providing important info for you. You just made our day (OSC team). Many thanks for doing such a awesome job! Please keep on providing feedback to us. It is very appreciated!

This will be a very long answer so brace yourself!

The metadata file is composed like this:

Metadata Parts	Explanation
HEADER	First row in the file. Has multiple formats that correspond to several BODY versions. (we call them metadata versions)
BODY	Multiple rows in the file. Contains timestamps, sensors and image/ video and image indexes.
FOOTER	Last row in the file. Was added after several versions in order to signal that a metadata file is complete.

Header format	Corresponding Medatada version(s) (BODY version)
device_model os_version	(we refer to this as) "no version"
device_model;os_version;metadata_version	version:1.0.1 version:1.0.2 version:1.0.3 version:1.0.4 version:1.0.5 version:1.0.6 version:1.0.7
device_model;os_version;metadata_version;app_version	version:1.0.8 version:1.1 version:1.1.1 version:1.1.2 version:1.1.3 version:1.1.4 version:1.1.5
device_model;os_version;metadata_version;app_version;rec_Type	version:1.1.6

Footer format	Metadata versions having Footer
"DONE"	version: 1.1.5 -> until current version

Metadata (BODY) version	Metadata (BODY) Format
'no version'	timestamp;longitude;latitude;elevation;horizontal_accuracy;gyroX;gyroY;gyroZ;accelerationX;accelerationY;accelerationZ;pressure;magneticX;magneticY;magneticZ;tripFrameIndex
'1.0.1'	timestamp;longitude;latitude;elevation;horizontal_accuracy;gyroX;gyroY;gyroZ;accelerationX;accelerationY;accelerationZ;pressure;compass;tripFrameIndex
'1.0.2'	timestamp;longitude;latitude;elevation;horizontal_accuracy;gyroX;gyroY;gyroZ;accelerationX;accelerationY;accelerationZ;pressure;compass;tripFrameIndex
'1.0.3'	timestamp;longitude;latitude;elevation;horizontal_accuracy;gyroX;gyroY;gyroZ;accelerationX;accelerationY;accelerationZ;pressure;compass;tripFrameIndex;gravityX;gravityY;gravityZ
'1.0.4'	timestamp;longitude;latitude;elevation;horizontal_accuracy;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compass;tripFrameIndex;gravityX;gravityY;gravityZ
'1.0.5'	timestamp;longitude;latitude;elevation;horizontal_accuracy;GPSspeed;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compass;tripFrameIndex;gravityX;gravityY;gravityZ
'1.0.6'	timestamp;longitude;latitude;elevation;horizontal_accuracy;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compass;tripFrameIndex;gravityX;gravityY;gravityZ
'1.0.7'	timestamp;longitude;latitude;elevation;horizontal_accuracy;GPSspeed;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compass;tripFrameIndex;gravityX;gravityY;gravityZ;OBD2speed
'1.0.8'	timestamp;longitude;latitude;elevation;horizontal_accuracy;GPSspeed;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compass;tripFrameIndex;gravityX;gravityY;gravityZ;OBD2speed
'1.1'	timestamp;longitude;latitude;elevation;horizontal_accuracy;GPSspeed;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compass;videoIndex;tripFrameIndex;gravityX;gravityY;gravityZ;OBD2speed
'1.1.1'	timestamp;longitude;latitude;elevation;horizontal_accuracy;GPSspeed;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compass;videoIndex;tripFrameIndex;gravityX;gravityY;gravityZ;OBD2speed
'1.1.2'	timestamp;longitude;latitude;elevation;horizontal_accuracy;GPSspeed;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compass;videoIndex;tripFrameIndex;gravityX;gravityY;gravityZ;OBD2speed
'1.1.3'	timestamp;longitude;latitude;elevation;horizontal_accuracy;GPSspeed;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compass;videoIndex;tripFrameIndex;gravityX;gravityY;gravityZ;OBD2speed
'1.1.4'	timestamp;longitude;latitude;elevation;horizontal_accuracy;GPSspeed;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compass;videoIndex;tripFrameIndex;gravityX;gravityY;gravityZ;OBD2speed
'1.1.5'	timestamp;longitude;latitude;elevation;horizontal_accuracy;GPSspeed;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compass;videoIndex;tripFrameIndex;gravityX;gravityY;gravityZ;OBD2speed;vertical_accuracy
'1.1.6'	timestamp;longitude;latitude;elevation;horizontal_accuracy;GPSspeed;yaw;pitch;roll;accelerationX;accelerationY;accelerationZ;pressure;compass;videoIndex;tripFrameIndex;gravityX;gravityY;gravityZ;OBD2speed;vertical_accuracy

As you can see from this table we had a few versions that are the same in format but are different as version number this is because there were some bugs on the Android and/or iOS apps that gather data.

Some remarks:

1. To get the best from OSC parse each version with the appropriate parser (we are using a map and we are using the correct format based on the HEADER->metadata_version)
2. If there are tacks in witch the API returns a number of photos and the metadata file does not or has a smaller/ larger number of photos may be because:
   • At the beginning the app did not create any metadata file only images. (the amount of images in this state should be very very small)
   • There were bugs that partially corrupted the metadata file (i can go into details but this is the long story short) (the amount of tracks having this issue should be also small)
   • Users or the app because of some problems may stop uploading a track before finishing the hole upload. (this can happen right now so we don't really know)
   • There still are some server bugs that can account for this cases. (this is happening right now)
3. The best you can do is use tripFameIndex from the metadata file and aggregate all sensors to a tripFrameIndex. Now you can use that tripFrameIndex to access the actual image from the API.

Heads up! tripFrameIndex from the metadata file can have gaps in indexes (this was caused by a bug on the clients in some old version). The API fixed this buy counting the occurrence of a tripFrameIndex row. eg: Lest simplify metadata a bit and write only the tripFrameIndex rows in buggy order (as it can be found in a metadata with issues). iPhone7Plus;10.3.2;1.1.5;1.5.8(1) 1499520707.044906;;;;;;;;;;;;;;0;0;;;;; 1499520708.380833;;;;;;;;;;;;;;0;1;;;;; 1499520721.434737;;;;;;;;;;;;;;0;10;;;;; 1499520728.970743;;;;;;;;;;;;;;0;11;;;;; 1499520741.901937;;;;;;;;;;;;;;0;19;;;;;

This should be like this. iPhone7Plus;10.3.2;1.1.5;1.5.8(1) 1499520707.044906;;;;;;;;;;;;;;0;0;;;;; 1499520708.380833;;;;;;;;;;;;;;0;1;;;;; 1499520721.434737;;;;;;;;;;;;;;0;2;;;;; 1499520728.970743;;;;;;;;;;;;;;0;3;;;;; 1499520741.901937;;;;;;;;;;;;;;0;4;;;;;

The API parses the metadata file and interprets it as it should be. So it will display only 5 images, but if you just read the last tripFrameIndex you would expect 19 images. This can also happen.

4. Using the iOS trips you should encounter less problems (I am the iOS developer that worked on this app 😄)

[patwater]

Wow thanks for this! We're digging in... Yes we've definitely had a seamless time with modern iOS trips so aligned there.

Also in the interest of helping out others perhaps worth posting some of this in formal documentation somewhere?

[vr00n]

Hi there. I have encountered a metadatafile that I need some help with.

The metadata file for trackId=1322833 looks like this.

I see that there is some metadata described at the head of the file but where can I find the headers for each of the different types. (Ex: Headers for GPS, OBD, DEVICEMOTION, COMPASS etc.)

Thanks!

METADATA:2.0
HEADER
ALIAS:d;DEVICE;1;1
ALIAS:g;GPS;1;1
ALIAS:o;OBD;1;1
ALIAS:m;DEVICEMOTION;1;1
ALIAS:p;PRESSURE;1;1
ALIAS:c;COMPASS;1;1
ALIAS:f;PHOTO;1;1
ALIAS:cam;CAMERA;1;1
ALIAS:exif;EXIF;1;1
BODY
1546715768.217877:d:iOS;iOS;12.1.2;iPhone11,8;2.5.1;1;video
1546715768.238706:cam:66.487946;189.0;1.7999999523162842
1546715768.1955886:m:1.5185647345324509;0.01493489571378024;-1.064857693273786;-0.0094260573387146;-0.004673212766647339;-0.01808035373687744;-0.8746224045753479;-0.014934340491890907;-0.48457464575767517
1546715768.2755895:m:1.515840179930235;0.009734123484157517;-1.062447004817247;-0.01495051383972168;0.0021045804023742676;-0.003612995147705078;-0.8735077381134033;-0.00973396934568882;-0.48671290278434753
1546715768.3155906:m:1.5158786010556633;0.008475382683824874;-1.0631813734564737;-0.010387182235717773;0.001471489667892456;-0.010337352752685547;-0.8738749027252197;-0.008475281298160553;-0.48607683181762695
1546715768.3555915:m:1.516981973445942;0.007640933142726338;-1.063872663878083;-0.004552245140075684;-0.003147125244140625;0.008074164390563965;-0.8742166757583618;-0.007640858646482229;-0.48547592759132385
1546715768.4355924:m:1.515568697754356;0.004259599280086246;-1.0614754157514508;-0.0002231001853942871;-0.0011115074157714844;-0.0005486011505126953;-0.8730679154396057;-0.004259586334228516;-0.4875800609588623
1546715768.4755936:m:1.514886267933967;0.002904837993462909;-1.0609497169396163;-0.004839301109313965;0.004445970058441162;-0.008357644081115723;-0.8728156685829163;-0.0029048339929431677;-0.48804131150245667
1546715766.8875868:p:99.74783325195312
1546715768.5155945:m:1.515406156018601;0.0026992574435407207;-1.0593149561891098;-0.000510871410369873;0.01017606258392334;0.007732570171356201;-0.8720172047615051;-0.0026992540806531906;-0.4894677996635437
1546715768.5955956:m:1.5155748561910627;0.0008267257219918302;-1.058179713891107;-0.001598358154296875;0.007403254508972168;0.013591349124908447;-0.8714637756347656;-0.0008267256198450923;-0.49045899510383606
1546715768.6355965:m:1.5157655994607768;-0.0009965888834992766;-1.0548873810317139;-0.008710980415344238;0.005183219909667969;-0.0001742839813232422;-0.8698443174362183;0.0009965887293219566;-0.49332547187805176
1546715768.6755974:m:1.5174278788890154;-0.002484575466047202;-1.0500007301083383;0.0019423961639404297;0.01225540041923523;-0.003560304641723633;-0.8674209117889404;0.0024845728185027838;-0.49756887555122375
1546715768.7555985:m:1.5183401926833822;-0.005822979036569339;-1.0417541701513602;0.019999265670776367;0.011663615703582764;-0.015138328075408936;-0.8632762432098389;0.0058229463174939156;-0.5046980977058411
1546715768.7955995:m:1.5172241625277243;-0.006463184373102803;-1.0441786668942186;-0.027778983116149902;-0.007134526968002319;0.006309568881988525;-0.8644939661026001;0.006463139317929745;-0.502601683139801
1546715768.8756006:m:1.5208041614837946;-0.003770066070392766;-1.0458040796454877;0.006129741668701172;0.018828213214874268;-0.009295344352722168;-0.865321695804596;0.003770057111978531;-0.5012027025222778
1546715768.9156015:m:1.5241583872622397;0.0016450928384057922;-1.0466250908980925;0.04650866985321045;0.04073810577392578;0.038245201110839844;-0.865737795829773;-0.0016450921539217234;-0.5004949569702148
1546715768.91414:g:37.91714097921525;-85.6671299614747;161.2649926904243;11.986149753858593;12.775282567818122;0.12572146952152252
1546715768.9453459:exif:1.8;4.25

[ciprianh-telenav]

@patwater, we plan to have the formal documentation published on openstreetcam.org website soon. Thank you for your patience. @vr00n, until the documentation will be published you may find the details on OpenStreetCam Metadata 2.0 Documentation.pdf

[patwater]

Thanks! Could you please let us know when the documentation is ready? Or will that be included on the OSM blog? http://blog.improve-osm.org/en/