[nebula] support downloaded without account (using bearer token)

xbjfk commented 2 years ago

Checklist

[X] I'm requesting a site-specific feature
[X] I've verified that I'm running yt-dlp version 2022.06.29 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[ ] I've read about sharing account credentials and I'm willing to share it if required

Region

Global?

Example URLs

https://nebula.app/videos/wendover-alaska https://nebula.app/videos/that-time-disney-remade-beauty-and-the-beast https://nebula.app/videos/jetlag-the-big-strategy-day-connect-4-across-america

Provide a description that is worded well enough to be understood

If you visit the URL, you can see the message "The first one is on us" where you can play the video in the browser with no account. After further digging, it turns out that the m3u8 is requested with ?token=[bearer token]. Running yt-dlp on the m3u8 (that I extracted with browser network tools) with ?token=[bearer token] downloads the video perfectly. When i say bearer token, I am talking about the one gotten by sending an empty POST to https://api.watchnebula.com/api/v1/authorization/ Currently, yt-dlp does not work on nebula without a username and password. EDIT: After further digging, yt-dlp seems to access zype directly, which I believe is not needed. A simple procedure I would use is: Get the video ID from input (jetlag-the-big-strategy-day-connect-4-across-america)

Get metadata:

https://content.watchnebula.com/video/jetlag-the-big-strategy-day-connect-4-across-america/ (with Authorization: Bearer [token]) (Note: this metadata sometimes contains "attributes: [ "free_sample_eligible" ]", but this does not necessarily mean this is the case, as discussed later. If free_sample_eligible is missing, the next call will return 401. wendover-alaska is an example of this)

Get playlist:

https://content.watchnebula.com/video/jetlag-the-big-strategy-day-connect-4-across-america/stream/ This returns JSON, containing: manifest: the m3u8 download: some URL of the video, typically low quality iframe: the zype iframe bif: bif files, seems to be a collection of thumbnails in a strage format subtitles: subtitles (also included in the m3u8)

For some videos, accessing these urls without login works fine, but for some, despite being "free_sample_eligible" they return error 403 with {"message":"This video requires a purchase entitlement in order to watch."}, and the zype iframe errors (this happens in the browser too). The disney video is an example of this, whereas connect 4 across america works perfectly

I am not sure how this procedure will work for real accounts, but it will probably be the same but with different authorization header.

Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://nebula.app/videos/that-time-disney-remade-beauty-and-the-beast']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.06.29 [9d339c4]
[debug] Python 3.10.5 (CPython 64bit) - Linux-5.18.4-gentoomy-kernel-x86_64-AMD_Ryzen_3_2200G_with_Radeon_Vega_Graphics-with-glibc2.35 (glibc 2.35)
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 4.4.2 (fdk,setts), ffprobe 4.4.2
[debug] Optional libraries: Crypto-3.14.1, brotli-1.0.9, certifi-3021.03.16, mutagen-1.45.1, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
[debug] Downloading _update_spec from https://github.com/yt-dlp/yt-dlp/releases/download/2022.06.29/_update_spec
Latest version: 2022.06.29, Current version: 2022.06.29
yt-dlp is up to date (2022.06.29)
[debug] [Nebula] Extracting URL: https://nebula.app/videos/that-time-disney-remade-beauty-and-the-beast
[Nebula] that-time-disney-remade-beauty-and-the-beast: Fetching video meta data
[Nebula] Reauthenticating to Nebula and retrying, because last bearer call resulted in error 403
ERROR: [Nebula] that-time-disney-remade-beauty-and-the-beast: This video is only available for registered users. Use --cookies, --cookies-from-browser, --username and --password, or --netrc (watchnebula) to provide account credentials
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 640, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/nebula.py", line 243, in _real_extract
    video = self._fetch_video_metadata(slug)
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/nebula.py", line 236, in _fetch_video_metadata
    return self._call_nebula_api(f'https://content.watchnebula.com/video/{slug}/',
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/nebula.py", line 82, in _call_nebula_api
    self._perform_login()
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/nebula.py", line 145, in _perform_login
    self._nebula_api_token = self._retrieve_nebula_api_token(username, password)
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/nebula.py", line 64, in _retrieve_nebula_api_token
    return self._perform_nebula_auth(username, password)
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/nebula.py", line 20, in _perform_nebula_auth
    self.raise_login_required()
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1106, in raise_login_required
    raise ExtractorError(msg, expected=True)

pukkandan commented 2 years ago

Related #4002

xbjfk commented 2 years ago

It seems like "This one's on us" is no longer a thing and you have to create and account

coletdjnz commented 2 years ago

It seems like "This one's on us" is no longer a thing and you have to create and account

Still seems to be a thing. It is based on browser cookies, so if you clear cookies/open in private browsing it'll work again.

Though it seems that not all videos have it (only 2/3 of the ones you provided do).

yt-dlp / yt-dlp