BT connection set's weird time stamps / recordings missing

EDsteve commented 4 months ago

There is a new issue, which might be serious if it happens more often. It seems on some devices recordings are missing and timestamps are wrong after a BT connection.

How it happens:

This ELOC was running under a tree near my house for 8 days. Almost every day i connect to it via BT from the app just to see if it is still recording. All seems fine in the app. Still shows recording. I never stopped the recording. I only connected with the app to check the status. When did i connect to the ELOC: 24 Feb: 10:36

Feb: 11:21
Feb: 13:59
Feb: 14:29
Feb: 13:20
Feb: 11:16 @LIFsCode @OOHehir These times corelate with the timestamp-change in the screenshots below.

Problem

When i look at the recordings from this ELOC after 8 days (when the battery was empty). I realized that it didn't record all the time and the timestamps don't make sense.

One hour recording is actually >61 minutes according to the timestamps in Windows explorer. But when i open one recording in Audacity it shows exactly 60 minutes. This accumulates over time which is a lot after one month recording. Might be connected to this issue?
The timestamp of each file seem to be using the time of the last BT connection. Untill i connect again.
Day 26 of February is missing completely

This is the /eloc content as txt file: BAd_TimeStamp.txt And the config: not_set1708625762254.txt

Here the screenshots (colored by day): time_off_0 time_off_1 time_off_2 time_off_3

Not sure if that is connected. But this ELOC always showed Battery state empty during these 8 days. Here is a getStatus:

ELOC without this issue:

And of course to make it more difficult. This does not always happen. Another ELOC also running outside did not show this issue. When i am testing ELCOs I normally only check if it has recordings and listen to a few of them, but i normally don't count them. That's why i can't say how often this happened already.

This is the folder content as txt from another ELOC without issues: Good_TimeStamps.txt And the config: black case1709055550864.txt

Would it make sense to fix the recording time, uptime issue? That would maybe also make it easier to realize these kind of problems?

OOHehir commented 4 months ago

@EDsteve @LIFsCode The origin of the times are: wav writer filename -> uses ESP32Time filesystem -> uses system (via fopen & stdio.h) As they're slightly separate sources this seems to point to an issue with the internal clock, as if it has stopped between BT connections. I'm not familiar wit the internal clock so will take some digging..

I can take a look at #77 in the meantime, should should be straightforward.

LIFsCode commented 4 months ago

@EDsteve @OOHehir

This is indeed a very complicated issue and I assume here are multiple issues mixing up in this:

[ ] 1. There are obviously missing files --> problem 1
[ ] 2. There is a (potentially) large timedrift in the timestamps, this may relate to https://github.com/LIFsCode/ELOC-3.0/issues/26 As mentioned in ffbfa34 the time drift of the oscillator is not yet evaluated. So maybe you just did the evaluation --> problem 2
[ ] 3. in blocks all files on the sd card have the same timestamp even though having different timestamps in their filename: This seems more a file system issue --> problem 3
[ ] 4. There might be an issue with the updating of the timestamps during runtime --> unproved so far

@EDsteve do you have any log files please?

Further steps to track the issue:

@EDsteve Can you redo the tests with larger logfiles. I have increased the default file size of the rotate files to 5 MByte 2633684 the 5 kByte end up to be far too small for effecient analysis.

@EDsteve Could you start the ELOC (which showed this errors) only once and not connect via BT and simple let it run for 1-2 days and then connect and stop it. Just to see how the timestamps drifted and keeping out all other issues. With this we can also have a look if there are any gaps within the record
Can you also redo the same test for multiple days with multiple BT reconnect with larger logfiles.

EDsteve commented 4 months ago

@LIFsCode

I have done some tests before about the timedrifts and could not find any timedrift within a few days. So the timedrift problem from this issue is either specific for only this one ELOC or some other reason. - > But will do more tests and report back
The timestamp in the filename is the time of creation and the time in windows explorer is date modified. So they should be different. Date modified should be 60 minutes behind date of creation in a working scenario. Not really sure if i understand you correctly. But in the screenshot they do have the same timestamp in filename and explorer (see picture below).

Log File: Unfortunately i tested this device with logs off.

I will do all tests soon and will report back.

And thanks for the recordingtime-fix. Will check it out as well @OOHehir

LIFsCode commented 4 months ago

@EDsteve Ah sorry I have read it wrong. So the files are 1 second apart even though they have samples for 60 seconds. That's strange

EDsteve commented 2 months ago

Got hopefully solved with #86

LIFsCode / ELOC-3.0