search

FalconLee1011 / Customized-Aerial-Screen-Saver

182 stars 7 forks source link

Balnk wallpaper #3

Open mitchjacksontech opened 9 months ago

mitchjacksontech commented 9 months ago

The desktop wallpaper often goes blank when using a custom arial set by this utility. It also is immediately blank when using the preview function. I have a theory about this. Here's some sippets from entries.json

One of the original buit-in video definitions

    {
      "localizedNameKey": "Y011_C001_0305_NAME",
      "showInTopLevel": true,
      "previewImage": "https:\/\/sylvan.apple.com\/itunes-assets\/Aerials126\/v4\/51\/ff\/08\/51ff0824-8da5-78f0-e218-9e61264965bb\/Landscapes_Yosemite_01@2x.png",
      "id": "4109D42A-D717-46A7-A9A2-FE53A82B25C0",
      "shotID": "Y011_C001_0305",
      "accessibilityLabel": "Yosemite",
      "pointsOfInterest": {
        "44": "Y011_C001_44",
        "120": "Y011_C001_120",
        "87": "Y011_C001_87",
        "585": "Y011_C001_585",
        "0": "Y011_C001_0",
        "390": "Y011_C001_390",
        "470": "Y011_C001_470"
      },
      "categories": [
        "A33A55D9-EDEA-4596-A850-6C10B54FBBB5"
      ],
      "preferredOrder": 50,
      "url-4K-SDR-240FPS": "https:\/\/sylvan.apple.com\/itunes-assets\/Aerials116\/v4\/cb\/5b\/50\/cb5b5035-6701-619f-9065-3d7d0e5fbef4\/Y011_C001_UHD_SDR_FRC240fps_sdr_4k_qp22_240p_t2160_tsa.mov",
      "subcategories": [
        "E4B726A1-7DD5-4E5A-A430-C99FE47E7A4F"
      ],
      "includeInShuffle": true
    },

Here's an custom item added by this utility

    {
      "shotID": "",
      "localizedNameKey": "Puget Sound",
      "pointsOfInterest": {},
      "showInTopLevel": true,
      "id": "3B6A19F0-DA52-4915-A7B8-DC47D37DF9E3",
      "accessibilityLabel": "Puget Sound",
      "subcategories": [
        "E2B12963-9698-43BE-890D-DDC900C95D00"
      ],
      "categories": [
        "C8EE7C2A-8025-4094-91C2-9637B6D2A64D"
      ],
      "url-4K-SDR-240FPS": "\/Users\/mjac\/Library\/Application Support\/com.xtl.customizedarealscreensaver\/video\/3B6A19F0-DA52-4915-A7B8-DC47D37DF9E3.mov",
      "preferredOrder": 0,
      "previewImage": "\/Users\/mjac\/Library\/Application Support\/com.xtl.customizedarealscreensaver\/asset-perview\/asset-preview-3B6A19F0-DA52-4915-A7B8-DC47D37DF9E3.jpg",
      "includeInShuffle": true
    }

This utility does not add a pointsOfInterest key. Is it possible when wallpaper gets set by the OS without actually playing the vdieo for a while and pausing on a frame, that it chooses a wallpaper from from the video using this pointsOfInterest array? Could these values be pointing at keyframes within the video for the OS to use as wallpaper images when setting wallpaper without actually playing through the video sequence? Maybe that's why the wallpaper keeps going blank? It looks like shotID may need a similar string identifying a frame from the video?

FalconLee1011 commented 9 months ago

@mitchjacksontech The theory you've provided seems possible, I will conduct some experiments based on this theory soon! thanks :))

Also, video itself might be a factor too, the recommended video format is: Attribute Requirements
Length At least 15 seconds
Audio Track Prefer no audio track
FPS (Frames Per Second) At 60 FPS
Aspect Ratio 16:9
Resolution At least 1920x1080
Video Format Must be .mov
ian-h-chamberlain commented 9 months ago

@FalconLee1011 that's a handy list of requirements, do you think it makes sense to add that to the README (or if possible, have the app check some of those things when user adds a video)?

I realize a lot of this is just reverse engineering Apple's implementation, so it makes sense that it's not straightforward to figure out what exactly is possible or not. But if you have seen experimentally that some of these things don't work then it would be great to document it!

jdrupal-dev commented 8 months ago

Any findings on this? I have the same issue, where the wallpaper goes black when logging in.

plambe commented 7 months ago

I tried adding pointsOfInterest, but it didn't help at all. They seem to be just offsets in seconds from the beginning of the video.

@FalconLee1011, where is the recommended video format documented?

I think the format might be the problem.

joelklint commented 5 months ago

Hi! 👋 I'm experiencing this issue aswell. Video works upon first screen lock. Upon first unlock everything goes black and stays black. Also the preview in System Preferences is broken from the very start.

I tried transcoding the video to 1920x1080 60 FPS and it still failed. The table below is describes the videos used. Attribute Requirements
Length 19 seconds
Audio Track Silence (not sure if video contains silent audio track or just no track at all)
FPS (Frames Per Second) 60 FPS
Aspect Ratio 16:9
Resolution 3840 × 2160 and 1920x1080
Video Format .mov
YusongLeng commented 5 months ago

Hi all,

I've been experiencing the 'blank wallpaper' issue as well, and have done some experiments to try and find a solution. Here's what I've discovered:

I started by downloading two official video wallpapers from System Settings:

  1. Landscape/California’s Temblor Range: F390FE3B-FA61-483D-BADC-2447F89951BA.mov
  2. Landscape/Redwoods from Above: AA5E82B9-289A-480C-A14B-242989107275.mov

I then swapped the filenames of these videos without altering their associated files or metadata (e.g. snapshots, entries.json). Interestingly, selecting 'California’s Temblor Range' in System Settings now plays the 'Redwoods from Above' video, and this wallpaper worked perfectly fine – the video smoothly stopped to a still wallpaper when I unlocked it back from Screensaver.

This led me to hypothesize that the 'blank wallpaper' issue lies within the video files themselves, rather than other system files.

With this assumption, my next step is to replicate the video specifications of the official wallpapers by encoding a new video. I first used ffprobe to extract the metadata of both official wallpapers, attached below:

  1. Landscape/California’s Temblor Range:

    Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Library/Application Support/com.apple.idleassetsd/Customer/4KSDR240FPS/F390FE3B-FA61-483D-BADC-2447F89951BA.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2023-08-18T03:36:16.000000Z
  Duration: 00:08:30.51, start: 0.000000, bitrate: 11946 kb/s
  Stream #0:0[0x1](und): Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt709), 3840x2160 [SAR 1:1 DAR 16:9], 11921 kb/s, 239.76 fps, 239.76 tbr, 239760 tbn (default)
    Metadata:
      creation_time   : 2023-08-18T03:36:16.000000Z
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
      encoder         : HEVC

  2. Landscape/Redwoods from Above:

    Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Library/Application Support/com.apple.idleassetsd/Customer/4KSDR240FPS/AA5E82B9-289A-480C-A14B-242989107275.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2023-08-14T17:52:43.000000Z
  Duration: 00:05:07.24, start: 0.000000, bitrate: 12593 kb/s
  Stream #0:0[0x1](und): Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt709), 3840x2160 [SAR 1:1 DAR 16:9], 12568 kb/s, 239.76 fps, 239.76 tbr, 239760 tbn (default)
    Metadata:
      creation_time   : 2023-08-14T17:52:43.000000Z
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
      encoder         : HEVC

Using this information, I created a dummy video to match these specs using ffmpeg. Here's the metadata of this dummy video:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'clips/slice_01.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf60.16.100
  Duration: 00:00:15.02, start: 0.000000, bitrate: 37607 kb/s
  Stream #0:0[0x1]: Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt709, progressive), 3840x2160 [SAR 1:1 DAR 16:9], 37583 kb/s, 239.76 fps, 239.76 tbr, 239760 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : FFMP
      encoder         : Lavc60.31.102 libx265

Unfortunately, this dummy video still encountered the 'blank wallpaper' issue, leading me to speculate that discrepancies in handler_name, vendor_id, encoder, or potentially unidentified metadata could be contributing factors.

The exact cause of the 'blank wallpaper' issue remains elusive. It suggests a deeper compatibility or encoding problem. Further investigation into metadata differences and additional encoding parameters may be necessary.

I hope these insights prove useful and look forward to any additional information or solutions from the community.

chrisly-bear commented 4 months ago

I'm experiencing the same issue. The wallpaper and screensaver work initially, but then go to black when moving from screensaver to desktop and stay that way.

I looked at the output of the Console.app and noticed these two errors:

error   17:22:27.426531+0200    WallpaperVideoExtension [<private>]: Error handling sample: Error Domain=WallpaperVideoCore.VideoSampleReadingErrors Code=4.
error   17:22:27.426595+0200    WallpaperVideoExtension [<private>]: VideoPlayerLayer failed to acquire next sample due to error: Error Domain=WallpaperVideoCore.VideoSampleReadingErrors Code=4

It sounds like the WallpaperVideoExtension tries to sample a frame from the video (presumably the frame that will be displayed as the static image of the desktop background?). I tried to figure out what the VideoSampleReadingError Code 4 means. The errors seem to be part of a private API, so there's no documentation on it. However, upon disassembling the WallpaperVideoExtension binary (/System/Library/ExtensionKit/Extensions/WallpaperVideoExtension.appex/Contents/MacOS/WallpaperVideoExtension) using Hopper I found the following enum:

enum WallpaperVideoCore.VideoSampleReadingErrors {
  case noAssetReader
  case noSampleBuffer
  case noPixelBuffer
  case noTimeAdjustedSampleBuffer
  case noTemporalInfo
}

Assuming these are in the correct order, error 4 would be noTemporalInfo, which does sound like there is some metadata missing in the video itself (I didn't find anything interesting in the Aerial.sqlite database, which could've potentially also stored such "temporal information"). However, comparing @YusongLeng's dummy video to one of the Apple videos, I don't see any glaring differences (at least using ffprobe, maybe there are unidentified metadata as YusongLeng also suggests?).

I came across this YouTube video that mentions the custom video must have the same length as the Apple video it replaces, so I tried replacing the "Tahiti Coast" video with a video of the exact same length (01:24 min). However, this lead to the same VideoSampleReadingError Code 4.

Not sure where to go from here, but hopefully this information is useful to someone looking into the problem (maybe someone with more reverse engineering or video encoding knowledge).

karashiiro commented 1 month ago

Also seeing this issue, but as a band-aid for others: I've found I can just use the customized Aerial as a screensaver only (unchecking "Show as wallpaper") and then setting the corresponding still image (the thumbnail, in my case) as my actual wallpaper. Might not look great for more complex Aerials, but it worked for the simple one I used.