owntracks / recorder

Store and access data published by OwnTracks apps
Other
907 stars 123 forks source link

What is the best way to get location data *in* to ot-recorder? #477

Closed gitcob closed 4 months ago

gitcob commented 4 months ago

First of all, I love the project! I've been using the android app with my own location tracking tinker projects for years and somehow just now discovered the web ui.

I save almost all of my MQTT messages in a database (mostly home automation stuff) and I'd love to look at that in the OwnTracks frontend as well. I have location data from other sources too, such as my sports watch and Google that could complete the picture.

In the docs I've seen plenty of ways to access the data from ot-recorder, but what is the best way to move lots of location data into it, ideally in bulk?

My idea is to just "re-play" all the OwnTracks MQTT messages I've recorded through my MQTT broker, and translate the other data sources to OT messages as well. Does that make sense or is there a more direct API that I've missed?

linusg commented 4 months ago

If you have GPX files this may be of use: https://github.com/owntracks/recorder/tree/master/contrib/gpx-to-rec

If you have overlapping .rec files they can be merged by concatenating the files (make sure to stop the recorder beforehand).

jpmens commented 4 months ago

what is the best way to move lots of location data into it, ideally in bulk?

In addition to what @linusg says, generally the .rec file format is quite trivial; if you look at the files, they contain a timestamp followed by a TAB, an asterisk, another TAB and then JSON containing at least the following elements in a single line (no newlines in the JSON):

{
  "_type": "location",
  "lat": 21.434299,
  "lon": -87.336965,
  "tid": "jp",
  "tst": 1704067151
}

The tid is a tracker ID (invent any two-character combination, e.g. your initials), tst is an epoch timestamp (which is the timestamp at which the position was recorded; the timestamp at the beginning of the line is when the message arrived at the Recorder).

2023-12-31T23:59:11Z    *   {"_type":"location","lat":21.434299,"lon":-87.336965,"tid":"jp","tst":1704067151}

So using whichever programming/scripting language you want, you might be able to dump your database locations into REC files.

If you then ensure your files are named YYYY-MM.rec you should be good to go.

gitcob commented 4 months ago

Thank you both, I think exporting my data in the .rec file format is perfect for my use case, seems straightforward enough!