.json location changed(?)

[MonkeyAlienGoose]

Looks like as of today the location for the .json has changed, as has the whole trailers page on apple.com, seems they are making it more Apple TV-facing and making back end changes while doing so -- getting "Error: could not load data from http://trailers.apple.com/trailers/home/feeds/just_added.json"

[mongy-code]

Yes its gone https://appleinsider.com/articles/23/08/31/apple-abandons-itunes-movie-trailers-site-and-ios-app Do we have an alternative?

[MonkeyAlienGoose]

Well, RIP. I've not found an alternative solution as of yet, but I'm sure with time something will surface.

Also just want to take this moment to thank Adam for the creation of the script! Countless hours have been spent with favorite trailers, with great quality, on repeat, so your work has been very much appreciated.

[aag]

Yes, it does seem that the Apple Trailers site is gone, and there are so few trailers on the Apple TV site that it's useless. Presumably it only has trailers for movies that are currently or will soon be available on Apple TV+.

I'll see if I can find another source for high-quality trailers and if I can update the script to use it. Since it will no longer be Apple Trailers I'll probably create a new repository, if I get something working, but I'll update this repository with a link to it.

[avioli]

Bummer. Thank you @aag for the solid tool.

[avioli]

Ok. I had some free time to look at this and getting the list of movies from the browser Dev Tools is fairly easy:

Visit https://trailers.apple.com/

Open up the Dev Tools Elements and find the list within a <script> tag with id shoebox-uts-api, then JSON decode its text content and get all the items from the canvas.shelves.items[].

Array.from(Object.entries(JSON.parse(document.querySelector('#shoebox-uts-api').textContent)))[0][1].canvas.shelves.reduce((agg, obj) => agg.concat(obj.items), [])

Sample movie object

```json { "genres": [ { "id": "umc.gnr.mov.thriller", "name": "Thriller", "type": "Genre", "url": "https://tv.apple.com/us/genre/thriller/umc.gnr.mov.thriller" } ], "id": "umc.cmc.2f5ymu2pjl4ltgii2r28388r9", "images": { "shelfImage": { "height": 2160, "isP3": false, "joeColor": "b:rgb(2,21,33) p:rgb(243,239,250) s:rgb(226,105,133) t:rgb(195,195,206) q:rgb(181,88,113)", "supportsLayeredImage": false, "url": "https://is5-ssl.mzstatic.com/image/thumb/Video126/v4/ef/50/c6/ef50c673-ca2a-d224-a261-8c9ba7746fd3/d3e8957f-d67d-49c6-91e0-a199ed0cd99a_TheGoodMother_iTunes_CoverArt_3840x2160.png/{w}x{h}.{f}", "width": 3840 }, "shelfImageBackground": { "height": 1080, "isP3": false, "joeColor": "b:rgb(75,65,57) p:rgb(246,228,192) s:rgb(248,214,143) t:rgb(212,196,165) q:rgb(213,184,126)", "supportsLayeredImage": false, "url": "https://is2-ssl.mzstatic.com/image/thumb/-bEtQPQ1bohtws62tqtzHA/{w}x{h}.{f}", "width": 1920 }, "transitionImage": { "height": 3240, "isP3": false, "joeColor": "b:rgb(0,3,12) p:rgb(15,189,243) s:rgb(3,161,232) t:rgb(12,151,197) q:rgb(2,129,188)", "supportsLayeredImage": false, "url": "https://is1-ssl.mzstatic.com/image/thumb/Features126/v4/9e/a3/7e/9ea37eee-c34b-4af9-0543-70160c76d3af/mzl.iclkwqpq.png/{w}x{h}sr.{f}", "width": 4320 } }, "releaseDate": 1693526400000, "secondaryActions": [ "AddToUpNext" ], "title": "The Good Mother", "type": "Movie", "url": "https://tv.apple.com/us/movie/the-good-mother/umc.cmc.2f5ymu2pjl4ltgii2r28388r9" } ```

After that - one can get the title and url from the movie object, then load each url to get the movie page, from which you can get the trailer video from the of:video meta. Unfortunately that's a m3u8 playlist and not a MOV file like before, so I had to use yt_dlp to download the video file.

Although very fragile based on the above - this is what I came up with:

#!/usr/bin/env python3

import os
import glob
import json
# import time
import requests
from bs4 import BeautifulSoup
from yt_dlp import YoutubeDL
from pathvalidate import sanitize_filename

req = requests.get('https://trailers.apple.com')
soup = BeautifulSoup(req.content, 'html.parser')
data = json.loads(soup.find(id='shoebox-uts-api').text)
shelves = data[list(data.keys())[0]]['canvas']['shelves']
movies = [item for obj in shelves for item in obj['items']]

yt_opts = {
    'format': 'bestvideo[ext=mp4]+bestaudio',
    'outtmpl': '__trailer__.mp4',
    'quiet': True,
    'progress': True,
}

for movie in movies:
    print(' --- Downloading: {}'.format(movie['title']))
    # if movie['releaseDate']:
    #     release_date = time.strftime('%Y-%m-%d', time.gmtime(movie['releaseDate']/1000))
    #     filename = sanitize_filename(f"{movie['title']}.{release_date}-trailer.mp4")
    # else:
    #    filename = sanitize_filename(f"{movie['title']}-trailer.mp4")
    filename = sanitize_filename(f"{movie['title']}-trailer.mp4")
    if os.path.isfile(filename):
        print(' --- Skipped - file exists')
        continue
    req = requests.get(movie['url'])
    soup = BeautifulSoup(req.content, 'html.parser')
    trailer_url = soup.find(property="og:video").attrs['content']
    # for a custom logger look at - https://github.com/yt-dlp/yt-dlp#adding-logger-and-progress-hook
    with YoutubeDL(yt_opts) as ydl:
        result = ydl.download([trailer_url])
        if result == 0:
            found = glob.glob('./__trailer__.*')
            if len(found):
                os.rename(found[0], filename)
                print(' --- Ok - {}'.format(filename))
            else:
                print(' --- Error - download not found')
        else:
            print(' --- Error - download failed')

requirements.txt

beautifulsoup4==4.12.2
pathvalidate==3.1.0
requests==2.31.0
yt-dlp==2023.7.6

Of course - this is super-simplified version of your script. It doesn't track what was downloaded, apart from file matching, and it doesn't put the files in a target directory. Nor has options for the video resolution. Although there are many options from the playlist file:

ID                                                  EXT RESOLUTION FPS │   FILESIZE   TBR PROTO │ VCODEC        VBR ACODEC     MORE INFO
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
audio-HE-stereo-64_vod-ak-amt.tv.apple.com-English  mp4 audio only     │                  m3u8  │ audio only        unknown    [en] English
audio-HE-stereo-64_vod-ap-amt.tv.apple.com-English  mp4 audio only     │                  m3u8  │ audio only        unknown    [en] English
audio-HE2-stereo-32_vod-ak-amt.tv.apple.com-English mp4 audio only     │                  m3u8  │ audio only        unknown    [en] English
audio-HE2-stereo-32_vod-ap-amt.tv.apple.com-English mp4 audio only     │                  m3u8  │ audio only        unknown    [en] English
audio-stereo-160_vod-ak-amt.tv.apple.com-English    mp4 audio only     │                  m3u8  │ audio only        unknown    [en] English
audio-stereo-160_vod-ap-amt.tv.apple.com-English    mp4 audio only     │                  m3u8  │ audio only        unknown    [en] English
290-0                                               mp4 556x232     24 │ ~  4.89MiB  290k m3u8  │ avc1.64001f  290k video only
290-1                                               mp4 556x232     24 │ ~  4.89MiB  290k m3u8  │ avc1.64001f  290k video only
383-0                                               mp4 628x262     24 │ ~  6.47MiB  384k m3u8  │ avc1.64001f  384k video only
383-1                                               mp4 628x262     24 │ ~  6.47MiB  384k m3u8  │ avc1.64001f  384k video only
537-0                                               mp4 704x294     24 │ ~  9.06MiB  538k m3u8  │ avc1.64001f  538k video only
537-1                                               mp4 704x294     24 │ ~  9.06MiB  538k m3u8  │ avc1.64001f  538k video only
790-0                                               mp4 776x324     24 │ ~ 13.32MiB  790k m3u8  │ avc1.64001f  790k video only
790-1                                               mp4 776x324     24 │ ~ 13.32MiB  790k m3u8  │ avc1.64001f  790k video only
1173-0                                              mp4 862x360     24 │ ~ 19.76MiB 1173k m3u8  │ avc1.64001f 1173k video only
1173-1                                              mp4 862x360     24 │ ~ 19.76MiB 1173k m3u8  │ avc1.64001f 1173k video only
1568-0                                              mp4 862x360     24 │ ~ 26.42MiB 1568k m3u8  │ avc1.64001f 1568k video only
1568-1                                              mp4 862x360     24 │ ~ 26.42MiB 1568k m3u8  │ avc1.64001f 1568k video only
2258-0                                              mp4 1186x496    24 │ ~ 38.04MiB 2258k m3u8  │ avc1.64001f 2258k video only
2258-1                                              mp4 1186x496    24 │ ~ 38.04MiB 2258k m3u8  │ avc1.64001f 2258k video only
2977-0                                              mp4 1484x620    24 │ ~ 50.16MiB 2978k m3u8  │ avc1.640020 2978k video only
2977-1                                              mp4 1484x620    24 │ ~ 50.16MiB 2978k m3u8  │ avc1.640020 2978k video only
3964-0                                              mp4 1910x798    24 │ ~ 66.78MiB 3964k m3u8  │ avc1.640028 3964k video only
3964-1                                              mp4 1910x798    24 │ ~ 66.78MiB 3964k m3u8  │ avc1.640028 3964k video only
5186-0                                              mp4 1910x798    24 │ ~ 87.37MiB 5186k m3u8  │ avc1.640028 5186k video only
5186-1                                              mp4 1910x798    24 │ ~ 87.37MiB 5186k m3u8  │ avc1.640028 5186k video only
6744-0                                              mp4 1910x798    24 │ ~113.62MiB 6745k m3u8  │ avc1.640028 6745k video only
6744-1                                              mp4 1910x798    24 │ ~113.62MiB 6745k m3u8  │ avc1.640028 6745k video only