New data file format

[dylanmikesell]

@mathedjoe and @ttsukianto, We are creating a new data logger with @arispes and would like this to be a seamless integration with animal tracker. Our new data format will have more than just GPS data. We will also have acceleration and gyroscrope data. @ajakef is able to write the data to whatever ASCII format we would like. How should we proceed? Should we use an existing format e.g.

Index, Date, Time, Latitude, Longitude, Altitude, Speed, Course, Distance, Type, Timeout, MSVs_QCN, Weight Criteria, SleepTime, EHPE, Satelite ID, Satelite
1, 2018/09/18, 16:36:53,0,0,18.67,0,0,0, -2(000000D2),15, B0,0,113,0,0,  -------- -------- -------- --------
2, 2018/09/18, 16:39:32,44.02327,-116.973518,664.899963,108,218,0, 00(00000000),160, C9, 3C,0,2704,1342179378,  -X-X---- -------- ----X--- --XX--X-
3, 2018/09/18, 17:04:39,44.023327,-116.973625,665.079956,0,358,10.649766, 00(00000000),9, B5, 3C,0,1616,1342179346,  -X-X---- -------- ----X--- ---X--X- 

or would you like us to just make a completely new format and then we create a new read subroutine for just this data format? I don't know the I/O routines in animal tracker well enough to know which approach is best. What are your thoughts?

[mathedjoe]

Exciting! The app is modularized to auto-detect the logger format using whatever unique structure we can find in the csv file (which we then read and clean with a custom function for each logger type).

So, I'd suggest logging the data into csv using whatever structure seems most efficient, and then just provide us with an example / template so we can write a reading function. For portability, I'd suggest adding a static header to every logged file with the list of field names in the first row of the csv.

On Mon, Apr 12, 2021 at 12:53 PM Dylan Mikesell @.***> wrote:

@mathedjoe https://github.com/mathedjoe and @ttsukianto https://github.com/ttsukianto, We are creating a new data logger with @arispes https://github.com/arispes and would like this to be a seamless integration with animal tracker. Our new data format will have more than just GPS data. We will also have acceleration and gyroscrope data. @ajakef https://github.com/ajakef is able to write the data to whatever ASCII format we would like. How should we proceed? Should we use an existing format e.g.

1, 2018/09/18, 16:36:53,0,0,18.67,0,0,0, -2(000000D2),15, B0,0,113,0,0, -------- -------- -------- -------- 2, 2018/09/18, 16:39:32,44.02327,-116.973518,664.899963,108,218,0, 00(00000000),160, C9, 3C,0,2704,1342179378, -X-X---- -------- ----X--- --XX--X- 3, 2018/09/18, 17:04:39,44.023327,-116.973625,665.079956,0,358,10.649766, 00(00000000),9, B5, 3C,0,1616,1342179346, -X-X---- -------- ----X--- ---X--X-```

or would you like us to just make a completely new format and then we create a new read subroutine for just this data format? I don't know the I/O routines in animal tracker well enough to know which approach is best. What are your thoughts?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mathedjoe/animaltracker/issues/75, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD6TBG66RDCSDHBEFRJFK3DTIM6SDANCNFSM42Z3F2EQ .

[dylanmikesell]

@ajakef, looks like you can write a CSV in whatever format you like. Just please make sure there is a header line and please write something up that describes the file format and contents (e.g. units and descriptions of columns). @mathedjoe and @ttsukianto should be able to get the file into AnimalTracker without much difficulty it sounds like.

[ajakef]

Copy. I'll document it well.

[ajakef]

Hi @mathedjoe, @ttsukianto, and @dylanmikesell, here is a sample data file. The header should be self-explanatory, but let me know if you have any questions. Note that most fields are logged twice per second continuously, but GPS-related fields are logged in cycles and only once per second. New files start every day at midnight UTC. sample_file.zip