Uploader - support upload frontier lab file format

[koonchaya]

Example files can be found from https://drive.google.com/drive/folders/1gU-p2q6GsypVFE3UQ6JFRhjgGeT_PIEE?usp=sharing

[carlybatist]

@koonchaya @antonyharfield from the email from Frontier Labs

Yes the timestamp is always in the filename with the following ISO-8601 format.

YYYYMMDDTHHMMSS

[YYYY] = 4 digit year

[MM] = zero-padded month

[DD] = zero-padded day

[HH] = zero-padded hour between 00 and 24.

[MM] = zero-padded minute between 00 and 59.

[SS] = zero-padded second between 00 and 59.

= offset from UTC +/-hhmm So for the filename: 20221030T074346+1000_REC [-27.50516+153.01890].wav The local time at the start of the recording was 7:43:46 am (in the time zone UTC+10:00) on the 30th of October 2022. Then there is an underscore “_”, followed by the schedule entry name (in this case “REC”), followed by the [GPS location] (in decimal degrees) of the recorder when it was last turned on. For the other various metadata implementations please see the attached text documents. The one that is standard across all recordings is at least the filenames. Only WAV file recordings that use the localisation firmware have the GUANO metadata implemented so far. And those filename have both a start and end time in ISO-8601 format but with micro-seconds in the time-stamps (so any clock drift/error can be corrected). i.e. S20221030T175959.697833+1000_E20221030T180133.693521+1000_-27.50517+153.01890.wav “S” for start time and “E” for end time. [Filename specs.txt](https://github.com/rfcx/arbimon-uploader/files/15338464/Filename.specs.txt) [FLAC metadata.txt](https://github.com/rfcx/arbimon-uploader/files/15338465/FLAC.metadata.txt) [GUANO.txt](https://github.com/rfcx/arbimon-uploader/files/15338466/GUANO.txt) [WAV LIST-INFO meta data.txt](https://github.com/rfcx/arbimon-uploader/files/15338467/WAV.LIST-INFO.meta.data.txt)

[naluinui]

@carlybatist It would also be useful to have the example files from frontier lab so that we can test it out.

[carlybatist]

@carlybatist It would also be useful to have the example files from frontier lab so that we can test it out.

@naluinui yes I've asked for them, waiting on a response. Will post them here once I get them

[carlybatist]

The user sent this as a follow-up, not sure if a fix for this can also be included in this ticket?

"There is one thing I’ve noticed with uploading the data for the solar bars in the arbimon uploader – the FL Solar bars data is sorted into daily folders, which I have been adding one at a time (I have this data also sorted into site folders, but if I add by site folder eg the higher folder level, the data isn’t recognised. I have to add by the lowest folder level). This would normally be fine, but I had 20 solar bars out for 6 months of continual recording, so there is A LOT of daily folders to upload. Is there a work around for this? I’m happy to do it, but it is pretty time consuming."

[rassokhina-e]

"There is one thing I’ve noticed with uploading the data for the solar bars in the arbimon uploader – the FL Solar bars data is sorted into daily folders, which I have been adding one at a time (I have this data also sorted into site folders, but if I add by site folder eg the higher folder level, the data isn’t recognised. I have to add by the lowest folder level). This would normally be fine, but I had 20 solar bars out for 6 months of continual recording, so there is A LOT of daily folders to upload. Is there a work around for this? I’m happy to do it, but it is pretty time consuming."

we can support only 1 level folder, so the user should just drop the dates folder each by each inside the site folder @carlybatist

[rassokhina-e]

[koonchaya]

@rassokhina-e When it is auto detect, can we make the filename timezone "timezone in filename" selected? The current one is not auto selected like audiomoth that auto selected the device configuration.

[rassokhina-e]

When it is auto detect, can we make the filename timezone "timezone in filename" selected? The current one is not auto selected like audiomoth that auto selected the device configuration.

I don't think so, because most of the time it’s site timezone, the format from the file has less priority. Most of our users are used site timezone, not the file timezone.

[koonchaya]

Just to clarify the timezone in filename, how is it different than site timezone? @rassokhina-e

[rassokhina-e]

there are different timezone parse, the site timezone we get from the site data, the file timezone we parse from the file name

[naluinui]

image from From https://github.com/rfcx/arbimon-uploader/issues/222#issuecomment-2125546791

💡Just an idea from my fresh eyes seeing this screenshot for the first time, would it be better to use the words "Auto-detect" instead of "timezone in filename" so that it's consistent with filename format?

[carlybatist]

💡Just an idea from my fresh eyes seeing this screenshot for the first time, would it be better to use the words "Auto-detect" instead of "timezone in filename" so that it's consistent with filename format?

I can't access the image?

[koonchaya]

@naluinui I wasn't sure about this either. We may have to discuss a bit more for additional requirement.

[naluinui]

Definitely. @carlybatist was created a task for it at https://github.com/rfcx/arbimon/issues/2006