Timelapse not viewable on iOS devices

David-Jackson commented 10 months ago

Environment

Camera: RPi HQ
OS: Bullseye
Allsky version: v2023.05.01_03
Raspberry Pi 4B 4GB

Bug Description

I have not been able to view timelapses created by AllSky on my iOS devices. All timelapses are generated properly, there are no errors in the log, Windows and Android devices are able to load the timelapses without issue, but iOS devices fail to load the video. I'm assuming this is a codec issue? Even though I thought h264 was playable by iOS devices.
I have been able to reproduce this issue on several iOS devices (one iPhone 14 Pro and two different iPad Pros) and browsers (Chrome and Safari).
The issue occurs on a fresh install with no changes to AllSky.
Screenshots:

iPhone	Android
iPhone timelapse on load screen	Android timelapse on load screen
iPhone timelapse playing screen	Android timelapse playing screen

Timelapse Settings in `config.sh`

########## Timelapse
TIMELAPSE="true"
TIMELAPSEWIDTH=0
TIMELAPSEHEIGHT=0
TIMELAPSE_BITRATE="2000k"
FPS=25
VCODEC="libx264"
PIX_FMT="yuv420p"
FFLOG="warning"
KEEP_SEQUENCE="false"
TIMELAPSE_EXTRA_PARAMETERS=""
UPLOAD_VIDEO="false"
TIMELAPSE_UPLOAD_THUMBNAIL="true"

Log / configuration files

No logs to upload, as there are no warnings or errors to report.

Alex-developer commented 10 months ago

David

Do you know what version of IOS is on each of the devices? If you can get the versions then I can run a few tests to see if I can see whats going wrong

Thanks

Alex

David-Jackson commented 10 months ago

Alex,

I'm using iPadOS 16.2 and iOS 17.1.2

4358haltern commented 9 months ago

I have the same problem.. The latest ios is installed everywhere

EricClaeys commented 9 months ago

Since this only happens on a specific iOS release I don't think it's related to Allsky. Can you try copying a video to your phone, then playing it from your phone to see if the problem is with the web or something else?

flewid commented 8 months ago

I too was experiencing this, so did a bit of digging - apparently this has 'been a thing' on ios for almost a decade.. so, I ended up adding this to my config, and now my iphone friends say they can view the videos!

TIMELAPSE_EXTRA_PARAMETERS="-level 3.1 -refs 11"

Also I should note, I also no longer have any artifacts in the video, so this was a double win.

EricClaeys commented 8 months ago

@flewid. Thanks a bunch. I will add this to the Troubleshooting documentation.

Do you see artifacts when not using those settings? If so, can you please try using just one of the settings, then just the other to see which one gets rid of the artifacts?

cd ~/allsky/images/DATE
# replace DATE with the date of a video you know had artifacts
# change config.sh to just one setting
timelapse.sh --output allsky-DATE-SETTING1_NAME.mp4 DATE
# change to the other setting
timelapse.sh --output allsky-DATE-SETTING2_NAME.mp4 DATE
# change to NO setting
timelapse.sh --output allsky-DATE-no-settings.mp4 DATE

You will end up with 4 videos in that directory - the original with both settings plus two with one setting each plus one with neither setting. If you can find a frame with artifacts and do a screenshot of all 4 files that would be fantastic, and if you don't mind I can include them in the documentation. Thanks

4358haltern commented 8 months ago

I too was experiencing this, so did a bit of digging - apparently this has 'been a thing' on ios for almost a decade.. so, I ended up adding this to my config, and now my iphone friends say they can view the videos!

TIMELAPSE_EXTRA_PARAMETERS="-level 3.1 -refs 11"

Also I should note, I also no longer have any artifacts in the video, so this was a double win.

This works❗️, thank you very much

flewid commented 8 months ago

Do you see artifacts when not using those settings? If so, can you please try using just one of the settings, then just the other to see which one gets rid of the artifacts?

Sure thing @EricClaeys , so I did some experimentation today and exported all four videos, including the 'time' command so we could see if any were quicker than the others (rpi 4, 4gb ram).

One thing to also note - playing these videos on desktop - (linux, firefox) - there are no artifacts on any of them, however, as noted below, playing on Android I do see the artifacts, using firefox as well. Not sure what's up there, but adding both options does appear to fix that for me completely.

Also when the artifacts do appear, it seems to be during a 'light change' -- day to night, or, where I have it placed for testing it gets light out of my front windows if they are on, and it APPEARS to happen whenever the lights are flipped on/off - on the ones that display artifacts.

NEITHER OPTION USED: real 19m11.669s user 65m20.155s sys 0m16.074s

file size: 116MB / 121377605 bytes artifacts: yes plays on ios: no

JUST "-refs 11": real 43m26.804s user 153m26.065s sys 0m24.467s

file size: 116 MB / 121417932 bytes artifacts: yes plays on ios: no

JUST "-level 3.1": real 14m18.680s user 47m57.623s sys 0m13.232s

file size: 116 MB / 121078268 bytes artifacts: none ! plays on ios: yes

BOTH "-level 3.1 -refs 11": real 43m39.879s user 154m27.676s sys 0m23.461s

file size: 116 MB / 121442286 bytes artifacts: none ! plays on ios: yes

And here are some screens of the artifacts I'm seeing on Android

5 png

EricClaeys commented 8 months ago

@flewid, thanks a TON! It looks like just -level 3.1 is needed? What operating system do you have on the Pi? Before I add it to the next release I want to make sure it works on Buster, Bullseye, and Bookworm.

FYI, many programs that create videos add a full image followed by some number of images that only contain changes to the last full image, followed by a full image, then changes, etc. This can drastically decrease the file size for scenes that don't change a lot. When someone turns on a light the whole scene changes so the next few "difference" frames are huge. I don't know exactly how this works but I suspect it's related to what you are seeing. Slower devices like phones and non-optimized browsers may have trouble keeping up so you see artifacts - usually blocking ss since the software often breaks images into blocks and works on them.

EricClaeys commented 8 months ago

Forgot to say, I can't find any documentation on the -refs or -level options to ffmpeg. How did you find out about them?

flewid commented 8 months ago

Forgot to say, I can't find any documentation on the -refs or -level options to ffmpeg. How did you find out about them?

I actually found it from this post from 2013 -- i only added the refs because he said he needed it to work, but yeah I've been using just level 3.1 and appears fine this way, and much quicker to generate the videos :)

I'm on raspbian - Debian 12.5 (bookworm?)

Makes sense on the frame stuff and the scene change, that's kind of what i was figuring myself, thanks for the explanation :)

So far, so good with these changes on my system, and android, ios, ipados, and desktop are all working with no artifacts !

EricClaeys commented 8 months ago

Thanks. That's Bookworm. I have Buster and Bullseye so will test there and hopefully put in the next release as a standard option so people don't need to add to TIMELAPSE_EXTRA_PARAMETERS. Thanks again for your help.

EricClaeys commented 8 months ago

@aaronwmorris, do you see any problems with using -level 3.1 with ffmpeg? The default is level 4 but Apple devices don't support that.

flewid commented 8 months ago

Thanks. That's Bookworm. I have Buster and Bullseye so will test there and hopefully put in the next release as a standard option so people don't need to add to TIMELAPSE_EXTRA_PARAMETERS. Thanks again for your help.

no problem at all, let me know if you need me to test anything else, happy to help :)

David-Jackson commented 8 months ago

@flewid Thanks! This fix worked perfectly.

EricClaeys commented 8 months ago

@David-Jackson, are you using just -level 3.1 ? And what operating system (run cat /etc/os-release)?

Eric

aaronwmorris commented 8 months ago

I do not see any isses with -level 3.1 other than perhaps the limits on resolution.

According to https://en.wikipedia.org/wiki/Advanced_Video_Coding Level 3.1 is limited to the following

720×480@80.0 (13)
720×576@66.7 (11)
1,280×720@30.0 (5)

The limits in the spec may not be hard limits, but I will admit I do not have a lot of media encoding experience.

flewid commented 8 months ago

I'm not sure if this is related, but since I made this change, now the star trail images seem to just be a singular image, not a star trail? kind of odd.

EricClaeys commented 8 months ago

@flewid, it's highly unlikely they are related. The -level 3.1 option doesn't change the images, just the video created by the images. See the documentation page on Startrails for ideas on how to "fix" the startrails problem. It could be that your images are slightly brighter with the full moon and they are now over the threshold.

flewid commented 8 months ago

thanks you're right, was due to the moon !

EricClaeys commented 4 months ago

Closing Issue. Using -level 3.1 is documented in the next Allsky release.

AllskyTeam / allsky