gopro / labs

GoPro Labs
Apache License 2.0
474 stars 39 forks source link

Hero 10 Very Long Time-lapses taking extra long gaps and deviating from the set interval #578

Open La-te opened 11 months ago

La-te commented 11 months ago

I've been running very long time-lapses with 10 minute interval. For some reason after running 2-4 h there is a 12-24 gap in the capture. I can see that from the photos taken and also from the screen stating it is still for example 11.2 h till the next photo. Why this happens and is there something to do about it? HERO10 Black v1.50.70

Couple of examples from the log with bolds on the strange parts when there is a long gap. Looks like there is then also missing some expected log entries when comparing to the other sequences of actions.

[2023-09-30 21:22:43] Woke with Alarm:[21:22m01]!NQmPN!S!1R [2023-09-30 21:22:47] Labs:Main Start [2023-09-30 21:22:48] Labs:Initialize [2023-09-30 21:22:48] Woke with Command:[21:22m01]!NQmPN!S!1R [2023-09-30 21:22:48] Start Now:Past by 48 seconds [2023-09-30 21:22:48] Timer wake command:!NQmPN!S!1R [2023-09-30 21:22:57] Shutdown and Sleep:580 seconds [2023-09-30 21:22:59] shutdown:rtc wake scheduled [2023-09-30 21:32:42] Woke with Alarm:[21:32m01]!NQmPN!S!1R [2023-09-30 21:32:46] Labs:Main Start [2023-09-30 21:32:47] Labs:Initialize [2023-09-30 21:32:47] Woke with Command:[21:32m01]!NQmPN!S!1R [2023-09-30 21:32:47] Start Now:Past by 47 seconds [2023-09-30 21:32:47] Timer wake command:!NQmPN!S!1R [2023-09-30 21:32:56] Shutdown and Sleep:580 seconds [2023-09-30 21:32:57] shutdown:rtc wake scheduled [2023-09-30 21:42:40] Woke with Alarm:[21:42m01]!NQmPN!S!1R [2023-10-01 01:50:52] Labs:Main Start [2023-10-01 01:50:53] Labs:Initialize [2023-10-01 01:50:53] Woke with Command:[21:42m01]!NQmPN!S!1R [2023-10-01 01:50:56] Shutdown and Sleep:25195 seconds [2023-10-01 01:50:58] shutdown:rtc wake scheduled

[2023-09-28 06:16:15] Shutdown and Sleep:580 seconds [2023-09-28 06:16:17] shutdown:rtc wake scheduled [2023-09-28 06:25:59] Woke with Alarm:[06:26m01]!NQmPN!S!1R [2023-09-28 06:26:03] Labs:Main Start [2023-09-28 06:26:04] Labs:Initialize [2023-09-28 06:26:04] Woke with Command:[06:26m01]!NQmPN!S!1R [2023-09-28 06:26:04] Start Now:Past by 4 seconds [2023-09-28 06:26:04] Timer wake command:!NQmPN!S!1R [2023-09-28 08:51:30] Labs:Main Start [2023-09-28 08:51:30] Labs:Initialize [2023-09-28 08:52:12] Labs:Main Start [2023-09-28 08:52:13] Labs:Initialize [2023-09-28 08:52:40] Labs:Main Start [2023-09-28 08:52:41] Labs:Initialize [2023-09-28 08:56:04] Woke with POWER: [2023-09-28 08:56:09] Labs:Main Start [2023-09-28 08:56:09] Labs:Initialize

[2023-09-28 11:32:23] Read QR:!585NQmPN!S!1R [2023-09-28 11:32:27] Shutdown and Sleep:580 seconds [2023-09-28 11:32:28] shutdown:rtc wake scheduled [2023-09-28 11:42:11] Woke with Alarm:[11:42m01]!NQmPN!S!1R [2023-09-28 11:42:15] Labs:Main Start [2023-09-28 11:42:16] Labs:Initialize [2023-09-28 11:42:16] Woke with Command:[11:42m01]!NQmPN!S!1R [2023-09-28 11:42:16] Start Now:Past by 16 seconds [2023-09-28 11:42:16] Timer wake command:!NQmPN!S!1R [2023-09-28 11:42:25] Shutdown and Sleep:580 seconds [2023-09-28 11:42:26] shutdown:rtc wake scheduled [2023-09-28 11:52:09] Woke with Alarm:[11:52m01]!NQmPN!S!1R [2023-09-28 11:52:13] Labs:Main Start [2023-09-28 11:52:14] Labs:Initialize [2023-09-28 11:52:14] Woke with Command:[11:52m01]!NQmPN!S!1R [2023-09-28 11:52:14] Start Now:Past by 14 seconds [2023-09-28 11:52:14] Timer wake command:!NQmPN!S!1R [2023-09-28 11:52:23] Shutdown and Sleep:580 seconds [2023-09-29 00:02:34] Labs:Main Start [2023-09-29 00:02:34] Labs:Initialize [2023-09-29 00:02:34] Woke with Command:[12:02m01]!NQmPN!S!1R [2023-09-29 00:02:37] Shutdown and Sleep:25195 seconds [2023-09-29 00:02:39] shutdown:rtc wake scheduled [2023-09-29 07:02:37] Woke with Alarm:[12:02m59]!NQmPN!S!1R [2023-09-29 07:02:41] Labs:Main Start [2023-09-29 07:02:42] Labs:Initialize [2023-09-29 07:02:42] Woke with Command:[12:02m59]!NQmPN!S!1R [2023-09-29 07:02:45] Shutdown and Sleep:17953 seconds [2023-09-29 07:02:45] shutdown:rtc wake scheduled [2023-09-29 12:02:01] Woke with Alarm:[12:02m30]!NQmPN!S!1R [2023-09-29 12:02:05] Labs:Main Start [2023-09-29 12:02:06] Labs:Initialize [2023-09-29 12:02:06] Woke with Command:[12:02m30]!NQmPN!S!1R [2023-09-29 12:02:06] Start Now:Past by 6 seconds [2023-09-29 12:02:06] Timer wake command:!NQmPN!S!1R [2023-09-29 12:02:15] Shutdown and Sleep:580 seconds [2023-09-29 12:02:16] shutdown:rtc wake scheduled [2023-09-29 12:11:59] Woke with Alarm:[12:11m01]!NQmPN!S!1R [2023-09-29 12:12:03] Labs:Main Start [2023-09-29 12:12:04] Labs:Initialize [2023-09-29 12:12:04] Woke with Command:[12:11m01]!NQmPN!S!1R [2023-09-29 12:12:04] Start Now:Past by 64 seconds [2023-09-29 12:12:04] Timer wake command:!NQmPN!S!1R [2023-09-29 12:12:13] Shutdown and Sleep:580 seconds [2023-09-29 12:12:14] shutdown:rtc wake scheduled

dnewman-gpsw commented 11 months ago

It is know issue that when using the wake timer, somethings can go wrong. I think it is slowly improving, Although I haven't done extensive testing of this type on HERO12 yet, it is known issue on all models.

[2023-09-30 21:42:40] Woke with Alarm:[21:42m01]!NQmPN!S!1R [2023-10-01 01:50:52] Labs:Main Start

[2023-09-30 21:42:40] Woke with Alarm:[21:42m01]!NQmPN!S!1R [2023-10-01 01:50:52] Labs:Main Start

These make it seem like the logging is not reporting some camera actions.

[2023-09-28 11:52:23] Shutdown and Sleep:580 seconds [2023-09-29 00:02:34] Labs:Main Start [2023-09-29 00:02:34] Labs:Initialize [2023-09-29 00:02:34] Woke with Command:[12:02m01]!NQmPN!S!1R [2023-09-29 00:02:37] Shutdown and Sleep:25195 seconds [2023-09-29 00:02:39] shutdown:rtc wake scheduled [2023-09-29 07:02:37] Woke with Alarm:[12:02m59]!NQmPN!S!1R

This might be a Labs math error. Sleep 580 at 11:52:23, will wake just past midnight, yet it though it was half a day off, as the next two sleeps 25195 and 17953 is pretty close to 12 hours. That should be looked at.

La-te commented 11 months ago

Sometimes I see it being close to 22 or 24 h of waiting time. I think I have never succeeded for longer than 3-4 h of time-lapse before it sets something much longer. Due to that 12/24 h I tried to switch the camera from 24 h to 12 h format and from RAW to Standard (jpg) but those didn't help either.

Is there anything I could do to help debug this? I haven't reset the settings to default but I don't think my settings are in anyway exotic.

dnewman-gpsw commented 11 months ago

Are you running without an internal battery, as that reduces reliability? With long form time-lapse with internal battery is typically pretty reliable, running for a full charge.

La-te commented 11 months ago

I've been using fully charged Enduro battery without external power.

The main motivation for me to get this device was to run long time-lapses on battery.

La-te commented 11 months ago

Maybe I'll reset the settings and try with the standard battery and let you know.

dnewman-gpsw commented 11 months ago

Enduro is fine. I do find the behavior odd, as these errors as more common with external power, and with a 10-minute interval you should easily do 24+ hr captures on battery. I've been running this exact script on a HERO12 (as I hadn't tested this on 12) and it is still running after 24 hours, and the Labs code for this hasn't changed in a long time.

I don't think that there is much that can be done, as the script is fine. A Labs !RESET may help, just incase you have something that randomly conflicting. You can use !MDBGL=1 for more logging, but it will not likely explain why the camera forgetting to wake.

I do have this script running on a HERO10 now. I will let you know if I experience an error.

dnewman-gpsw commented 11 months ago

My HERO10 ran a fairly long time, but did stop before the battery was flat with random freeze (well the logging just stopped, and the camera was off when I checked.) This randomness is why don't recommend this style of long form timelapse, at least on critical projects without some type external reset. The two HERO12s I was running are still going, but this a random error, so they might be affected at some point.

dnewman-gpsw commented 11 months ago

Run HERO10 again with a 2 minute interval, and it run until battery ended. Same for HERO11. HERO12 with 10 minute interval is still going, 17% battery.

La-te commented 11 months ago

Now it worked till battery was empty after almost two days. I did !RESET, formatted the sd card on Hero10 and put the standard battery in. I didn't change any settings and put it running and it ran without gaps. Next I'll change to Enduro and adjust some settings. Hopefully it will run smoothly.

dnewman-gpsw commented 11 months ago

HERO12 finally ran out of battery, ran for ~4 days, using a 10 minute interval.

La-te commented 10 months ago

It hasn't been reliable anymore. At first, it seemed like working when choosing standard photo. However, after choosing RAW the issue came back. Even after choosing standard photo again there are issues. I'm wondering is it a problem when taking RAW photos or otherwise having "large" number of files like 1500 - 2000 photos. As I shared earlier I had success with empty card and taking only standard photos. I'm using SanDisk Extreme 128 GB U-3 A2 which I expect to be fine for the purpose. I have other brands with similar specs so I'll try them as well now. However, this would not be the first time when one of my action cameras starts to struggle with high number of files.

One interesting thing I saw is the following. When I go and push the side button to see is the time-lapse running anymore it might have stopped and reports for example that it will wait 20.3 h or 6.5 h till the next photo. However, when I see the log file now the reported delay is there always something else. Most of the time it is 25195 seconds so around 7 h. Couple of times it has been 17953 seconds. Below you can see the log.

17:45:54 the time-lapse has stopped 23:05:12 I press the side button to see if it is running or not. It displays some long delay but in the log I see "Shutdown and Sleep:25195 seconds" and decide to cancel and make a new time-lapse

[2023-10-17 17:41:13] Woke with Alarm:[17:41m01]!NQmPN!S!1R [2023-10-17 17:41:17] Labs:Main Start [2023-10-17 17:41:18] Labs:Initialize [2023-10-17 17:41:18] Woke with Command:[17:41m01]!NQmPN!S!1R [2023-10-17 17:41:18] Start Now:Past by 18 seconds [2023-10-17 17:41:18] Timer wake command:!NQmPN!S!1R [2023-10-17 17:41:25] Shutdown and Sleep:251 seconds [2023-10-17 17:41:26] shutdown:rtc wake scheduled [2023-10-17 17:45:40] Woke with Alarm:[17:45m01]!NQmPN!S!1R [2023-10-17 17:45:44] Labs:Main Start [2023-10-17 17:45:45] Labs:Initialize [2023-10-17 17:45:45] Woke with Command:[17:45m01]!NQmPN!S!1R [2023-10-17 17:45:45] Start Now:Past by 45 seconds [2023-10-17 17:45:45] Timer wake command:!NQmPN!S!1R [2023-10-17 17:45:54] Shutdown and Sleep:251 seconds [2023-10-17 23:05:12] Labs:Main Start [2023-10-17 23:05:12] Labs:Initialize [2023-10-17 23:05:12] Woke with Command:[17:50m01]!NQmPN!S!1R [2023-10-17 23:05:15] Shutdown and Sleep:25195 seconds [2023-10-17 23:05:17] shutdown:rtc wake scheduled [2023-10-17 23:05:34] Labs:Main Start [2023-10-17 23:05:35] Labs:Initialize [2023-10-17 23:05:35] Woke with Command:[17:50m59]!NQmPN!S!1R [2023-10-17 23:05:37] Cancel:CANCEL TIME DELAY [2023-10-17 23:05:38] Timer wake command: [2023-10-17 23:07:50] Read QR:!302NQmPN!S!1R [2023-10-17 23:07:54] Shutdown and Sleep:297 seconds [2023-10-17 23:07:55] shutdown:rtc wake scheduled [2023-10-17 23:12:55] Woke with Alarm:[23:12m01]!NQmPN!S!1R [2023-10-17 23:12:59] Labs:Main Start [2023-10-17 23:13:00] Labs:Initialize [2023-10-17 23:13:00] Woke with Command:[23:12m01]!NQmPN!S!1R [2023-10-17 23:13:00] Start Now:Past by 60 seconds [2023-10-17 23:13:00] Timer wake command:!NQmPN!S!1R [2023-10-17 23:13:16] Shutdown and Sleep:297 seconds [2023-10-17 23:13:17] shutdown:rtc wake scheduled [2023-10-17 23:18:16] Woke with Alarm:[23:18m01]!NQmPN!S!1R [2023-10-17 23:18:20] Labs:Main Start [2023-10-17 23:18:21] Labs:Initialize [2023-10-17 23:18:21] Woke with Command:[23:18m01]!NQmPN!S!1R [2023-10-17 23:18:21] Start Now:Past by 21 seconds [2023-10-17 23:18:21] Timer wake command:!NQmPN!S!1R [2023-10-17 23:18:37] Shutdown and Sleep:297 seconds [2023-10-17 23:18:38] shutdown:rtc wake scheduled

dnewman-gpsw commented 10 months ago

This is a more typical failure-to-wake error, and it is unlikely due to the amount of media or Raw mode. This is why I don't recommend this type of timelapse, as the wake alarm seems to have 0.1% failure rate on HERO10 (might be better on H12, as I've yet to see it fail this way), but this has been the case ever since Labs introduction. If you run a wake-sleep script for long enough time, it will forget to wake. When you press power, it reads the time it should have started, and then sets the alarm for the next day.

La-te commented 10 months ago

Seems to me that according to the log it is missing the rtc wake. Normally the last two entries in the log are the ones below. However, when the error occurs there is not the latter message "shutdown:rtc wake scheduled". Hence, it never wakes until I press the button and then schedules 25195 seconds sleep. This seems to be the case in 3-4 cases I looked from the log.

[2023-10-19 18:06:07] Shutdown and Sleep:132 seconds [2023-10-19 18:06:08] shutdown:rtc wake scheduled

La-te commented 10 months ago

I did !RESET!1O and !MDBGL=1 and attached the log. I don't see there anything more useful than with the normal log level. It did shutdown without rtc wake and after waking with the button the delay was 25195 seconds. This time though it resumed with the previously set interval when I woke it with the button the second time pretty much immediately.

[2023-10-20 20:35:33] Shutdown and Sleep:196 seconds [2023-10-20 20:54:14] Labs:Main Start [2023-10-20 20:54:14] Labs LCD ID:197 [2023-10-20 20:54:14] Naked when 0:1 [2023-10-20 20:54:14] Labs:Initialize [2023-10-20 20:54:14] Woke with Command:[20:38m01]!NQmPN!S!1R [2023-10-20 20:54:17] Shutdown and Sleep:25195 seconds [2023-10-20 20:54:19] shutdown:rtc wake scheduled [2023-10-20 20:54:30] Labs:Main Start [2023-10-20 20:54:30] Labs LCD ID:197 [2023-10-20 20:54:30] Naked when 0:1 [2023-10-20 20:54:30] Labs:Initialize [2023-10-20 20:54:30] Woke with Command:[20:38m59]!NQmPN!S!1R [2023-10-20 20:54:30] Start Now:Past by 990 seconds [2023-10-20 20:54:30] Timer wake command:!NQmPN!S!1R [2023-10-20 20:54:30] ACTION:!NQmPN!S!1R [2023-10-20 20:54:31] CAMMODE token:PN [2023-10-20 20:54:31] presets_edit:done [2023-10-20 20:54:32] ACTION:!S!1R [2023-10-20 20:54:35] ACTION:!1R [2023-10-20 20:54:38] ACTION:!201NQmPN!S!1R [2023-10-20 20:54:41] Shutdown and Sleep:196 seconds [2023-10-20 20:54:42] shutdown:rtc wake scheduled

qrlog.txt

If this is somewhat common issue has there been any discussion or decision so far whether a fix will be attempted or not? Is it potentially hardware related issue and hard to solve with software? I thought I could get two older Heros with the value of one Hero1 :)

dnewman-gpsw commented 10 months ago

This is not solvable with firmware/software, as not getting a wake message is at the hardware level. Well technically the a micro controller that has power and wakes the rest of the camera, Labs code only runs once the main camera has booted. This is why it could be better on HERO12, could be more robust code on the micro controller. For other cameras (and maybe H12 too) using an external timer and !MWAKE=2 to boot on power, has been the only reliable workaround.

La-te commented 10 months ago

Maybe my Hero 10 is a bad unit as failure rate is quite high. Higher than 0.1%. Sometimes it has taken only 1-4 photos, sometimes it runs for couple of hours. My Hero 9 seems to be working well. Is Hero 11 expected to be any better than (my) Hero 10?

dnewman-gpsw commented 10 months ago

I've found HERO8 and 12 to be the most reliable for repeated wake events. Completely anecdotal. Not enough testing on H12 yet.

kelp-videos commented 7 months ago

@dnewman-gpsw I am having a similar issue of my timelapse deviating from the actual time (I assume my gopro pauses its time when processing the photos?). Could you provide specific examples of how to set up an external timer for gopro to use and what timer you would reccomend?