[NiconicoLive] Unable to find stream media URL due to new Confirmation Code

hueyyeng commented 2 years ago

Checklist

[x] I'm reporting a broken site support issue
[x] I've verified that I'm running youtube-dl version 2020.12.14
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar bug reports including closed ones
[x] I've read bugs section in FAQ

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--write-info-json', '--continue', '--verbose', '--username', 'PRIVATE', '--password', 'PRIVATE', 'https://live.nicovideo.jp/watch/lv335747966']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2020.12.14
[debug] Git HEAD: 74c803336
[debug] Python version 3.8.9 (CPython) - Windows-10-10.0.19041-SP0
[debug] exe versions: ffmpeg N-79906-gc8c14d0
[debug] Proxy map: {'no': '192.168.99.100'}
[NiconicoLive] Logging in
[NiconicoLive] lv334138532: Downloading webpage
[NiconicoLive] Detected post-March 2019 HLS-based stream
ERROR: Unable to find stream media URL. Is the stream private or unavailable?
Traceback (most recent call last):
  File "I:\K_Drive\Library\Python\youtube-dl\youtube_dl\YoutubeDL.py", line 804, in wrapper
    return func(self, *args, **kwargs)
  File "I:\K_Drive\Library\Python\youtube-dl\youtube_dl\YoutubeDL.py", line 825, in __extract_info
    ie_result = ie.extract(url)
  File "I:\K_Drive\Library\Python\youtube-dl\youtube_dl\extractor\common.py", line 533, in extract
    ie_result = self._real_extract(url)
  File "I:\K_Drive\Library\Python\youtube-dl\youtube_dl\extractor\niconico.py", line 1057, in _real_extract
    raise ExtractorError('Unable to find stream media URL. Is the stream private or unavailable?', expected=True)
youtube_dl.utils.ExtractorError: Unable to find stream media URL. Is the stream private or unavailable?

Description

I've been using this forked of youtube-dl for the NiconicoLive support and been using it to archive drawing livestream although my last usage was late Dec 2021.

The issue is that Niconico seems to change the backend server side to dispatch a Confirmation Code whenever one try to login to the site and this includes this tool. With the Confirmation Code checks in place, youtube-dl will error out at youtube_dl.utils.ExtractorError: Unable to find stream media URL. Is the stream private or unavailable?

If in a browser, after inputting the Confirmation Code (which is a 6 digits code), you're login into the site and can watch the timeshift broadcast (basically the reason why I sub to Niconico for the archived broadcast).

I'm unsure how the login process works but I can take a look at the code and see if I can find a way to prompt the user to key in the Confirmation Code and proceed with the download. Just to report the issue here first if others are facing the same issue for NiconicoLive or Niconico in general.

hueyyeng commented 2 years ago

Follow up on this, I can't properly parsed the processed embedded data as it seems like it doesn't load the Confirmation Code page (likely the authentication function didn't detect the redirection).

What I noticed from the function logic is that it is looking for the following before it creates a new thread to handle the websocket connection:

websocket_url = embedded_data['site']['relive']['webSocketUrl']
best_quality = embedded_data['program']['stream']['maxQuality']

One can get the value above easily by login into Niconico and clear the Confirmation Code with your browser of choice:

Login into Niconico site and key in your Confirmation Code (either through 2FA app or the email from Niconico)
Right click and View Page Source (I'm using Firefox)
Search for webSocketUrl and you should see something like this: wss://a.live2.nicovideo.jp/unama/wsapi/v2/watch/RNGIDHERE/timeshift?audience_token=YOURLONGTOKENHERE
As for best_quality, you can use super_high since you need to have an active sub to Niconico to use it anyway.
Modify the niconico.py script (refer to screenshot) and run youtube_dl script
???
Profit

horahoradev commented 2 years ago

sorry for the late response, I don't look at this repo much; I'll work on adding updated nicolive support to yt-dlp in the future. nice art btw

horahoradev commented 2 years ago

@nolanlum Is it okay if I modify your nicolive code and add it to yt-dlp?

nolanlum commented 2 years ago

@nolanlum Is it okay if I modify your nicolive code and add it to yt-dlp?

sure! if you can adapt it to work in yt-dlp that's very welcome. thanks for keeping it up to date, let me know if there's anything I can do to help.

hueyyeng commented 2 years ago

@horahoradev If you need someone to QA, you can ping me as I'm currently subbing to Niconico for the timeshift feature.

animelover1984 / youtube-dl