search

mathedjoe / animaltracker

Visualize and analyze GPS data for cows (and other animals).
5 stars 6 forks source link

Columbus P-1 read capability #39

Closed dylanmikesell closed 4 years ago

dylanmikesell commented 4 years ago

Because we are now also using the Columbus P-1 GPS sensors, we need the ability to read RAW data from those sensors. The sensor outputs three formats: CSV, GPX (XML format), and TXT (NMEA format). Of the three formats, NMEA is the only output that provides information about the satellites and waypoint solution information (e.g. HDOP). I have attached an example NMEA file. Each line is a NMEA "sentence" and begins with the corresponding NMEA code (see bold below). Each data point has two NMEA sentences (RMC and GGA).

$GPRMC,192352.799,A,4336.0500,N,11611.8394,W,0.00,0.00,301019,,,A78 $GPGGA,192352.899,4336.0500,N,11611.8393,W,1,04,2.63,755.6,M,-18.6,M,,000058 $GPRMC,192352.899,A,4336.0500,N,11611.8393,W,0.00,0.00,301019,,,A70 $GPGGA,192353.000,4336.0500,N,11611.8391,W,1,04,2.64,755.8,M,-18.6,M,,00005A

We need to add functionality to AnimalTracker to read NMEA text files. We need a flag that first looks at the file extension. Then if it is a TXT file, it needs to read the first row of the first line and determine if the file is NMEA (e.g. is row 1, col 1 equal to $GPRMC). Then we need to parse the data associated with each waypoint. The attached PDF file explains the components of each NMEA sentence, and the colors in the example sentences highlight the important parameters that I think we will want to extract from the files.

NMEA_sentences.pdf

30192350.TXT

mathedjoe commented 4 years ago

@dylanmikesell - can you take a look at this processed version of your sample data file (rename to .csv to open in excel)? I think I got the important information, but we can easily adjust. sample_processed_data.txt

dylanmikesell commented 4 years ago

I think you're good to go. It looks like you got all of the important parts.

ttsukianto commented 4 years ago

When a user uploads data through the app, would I assume iGotU and Columbus P-1 datasets are in separate folders or could they potentially upload both types of data in the same folder?

dylanmikesell commented 4 years ago

I think they could be the same folder. We just need some distinguishing file name or header. You can store in separate folders if you prefer. But the two folders should then be under a data folder or something and have practical names related to the data.

On Fri, Nov 8, 2019 at 10:14 PM Thea Sukianto notifications@github.com wrote:

When a user uploads data through the app, would I assume iGotU and Columbus P-1 datasets are in separate folders or could they potentially upload both types of data in the same folder?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mathedjoe/animaltracker/issues/39?email_source=notifications&email_token=ABVIE4SI5PK6H7PLZW4F3Y3QSY2LBA5CNFSM4JHLHLFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDT5LOA#issuecomment-552064440, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABVIE4RL7CCDJYKKREATGW3QSY2LBANCNFSM4JHLHLFA .

mathedjoe commented 4 years ago

Typical use case might be all one format, but I think we should "peek" into every file, determine the format, and read/process accordingly. That may require a little refactoring of the code, but I think it's the right way to go and won't affect performance much.

Joe

On Sat, Nov 9, 2019 at 8:51 AM Dylan Mikesell notifications@github.com wrote:

I think they could be the same folder. We just need some distinguishing file name or header. You can store in separate folders if you prefer. But the two folders should then be under a data folder or something and have practical names related to the data.

On Fri, Nov 8, 2019 at 10:14 PM Thea Sukianto notifications@github.com wrote:

When a user uploads data through the app, would I assume iGotU and Columbus P-1 datasets are in separate folders or could they potentially upload both types of data in the same folder?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/mathedjoe/animaltracker/issues/39?email_source=notifications&email_token=ABVIE4SI5PK6H7PLZW4F3Y3QSY2LBA5CNFSM4JHLHLFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDT5LOA#issuecomment-552064440 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABVIE4RL7CCDJYKKREATGW3QSY2LBANCNFSM4JHLHLFA

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/mathedjoe/animaltracker/issues/39?email_source=notifications&email_token=AD6TBG77WYXHYFWZ7QCMTELQS3MADA5CNFSM4JHLHLFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDUI73A#issuecomment-552112108, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD6TBG7M6YHDMKXZXXKEIWDQS3MADANCNFSM4JHLHLFA .