Can't download from bangumi/bilibili pages using certain URLs

dxing97 commented 6 years ago

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2018.02.08. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2018.02.08

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones
[x] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

Single episodes using the URL I get from my browser:

$ youtube-dl -v https://www.bilibili.com/bangumi/play/ep63725 [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'-v', u'https://www.bilibili.com/bangumi/play/ep63725'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2018.02.08 [debug] Python version 2.7.12 (CPython) - Linux-4.4.0-43-Microsoft-x86_64-with-Ubuntu-16.04-xenial [debug] exe versions: none [debug] Proxy map: {} [generic] ep63725: Requesting header WARNING: Falling back on generic information extractor. [generic] ep63725: Downloading webpage [generic] ep63725: Extracting information ERROR: Unsupported URL: https://www.bilibili.com/bangumi/play/ep63725 Traceback (most recent call last): File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 2192, in _real_extract doc = compat_etree_fromstring(webpage.encode('utf-8')) File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 2542, in compat_etree_fromstring doc = _XML(text, parser=etree.XMLParser(target=_TreeBuilder(element_factory=_element_factory))) File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 2531, in _XML parser.feed(text) File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1653, in feed self._raiseerror(v) File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror raise err ParseError: not well-formed (invalid token): line 5, column 52 Traceback (most recent call last): File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 785, in extract_info ie_result = ie.extract(url) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 440, in extract ie_result = self._real_extract(url) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 3111, in _real_extract raise UnsupportedError(url) UnsupportedError: Unsupported URL: https://www.bilibili.com/bangumi/play/ep63725


But if I format the URL as described in [#11845](https://github.com/rg3/youtube-dl/issues/11845#issuecomment-276963207):

$ youtube-dl -v http://bangumi.bilibili.com/anime/2543/play#63725 [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'-v', u'http://bangumi.bilibili.com/anime/2543/play#63725'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2018.02.08 [debug] Python version 2.7.12 (CPython) - Linux-4.4.0-43-Microsoft-x86_64-with-Ubuntu-16.04-xenial [debug] exe versions: none [debug] Proxy map: {} [BiliBili] 63725: Downloading webpage [BiliBili] Downloading episode 63725. To download all videos in anime 2543, re-run youtube-dl with http://bangumi.bilibili.com/anime/2543 [BiliBili] 63725: Downloading JSON metadata [BiliBili] 63725: Downloading video info page [debug] Default format spec: best/bestvideo+bestaudio [debug] Invoking downloader on u'http://tx.acgvideo.com/a/df/3889896-1.mp4?txTime=1518209338&platform=pc&txSecret=1279e75c3a7605714c01116098f51c78&oi=2164441935&rate=60642&hfb=b99ffc3c5c68f00a33123bb25f882d5b' [download] Destination: 狐妖小红娘：番外-63725.mp4 [download] 100% of 16.60MiB in 03:47


Although this doesn't always work (error -5016, seen in [#14296](https://github.com/rg3/youtube-dl/issues/14296):

$ youtube-dl -v https://bangumi.bilibili.com/anime/2543/play#191521 [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'-v', u'https://bangumi.bilibili.com/anime/2543/play#191521'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2018.02.08 [debug] Python version 2.7.12 (CPython) - Linux-4.4.0-43-Microsoft-x86_64-with-Ubuntu-16.04-xenial [debug] exe versions: none [debug] Proxy map: {} [BiliBili] 191521: Downloading webpage [BiliBili] Downloading episode 191521. To download all videos in anime 2543, re-run youtube-dl with https://bangumi.bilibili.com/anime/2543 [BiliBili] 191521: Downloading JSON metadata [BiliBili] 191521: Downloading video info page ERROR: BiliBili returns error -5016 Traceback (most recent call last): File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 785, in extract_info ie_result = ie.extract(url) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 440, in extract ie_result = self._real_extract(url) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/bilibili.py", line 131, in _real_extract self._report_error(video_info) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/bilibili.py", line 82, in _report_error raise ExtractorError('%s returns error %d' % (self.IE_NAME, result['code']), expected=True) ExtractorError: BiliBili returns error -5016


---

### Description of your *issue*, suggested solution and other information

Explanation of your *issue* in arbitrary form goes here. Please make sure the [description is worded well enough to be understood](https://github.com/rg3/youtube-dl#is-the-description-of-the-issue-itself-sufficient). Provide as much context and examples as possible.
If work on your *issue* requires account credentials please provide them or explain how one can obtain them.

It looks like the in-browser URLs don't play nice with youtube-dl.

weituotian commented 6 years ago

https://www.bilibili.com/bangumi/play/ep173288 -> http://bangumi.bilibili.com/anime/21542/play#173288

gif

C:\Users\Administrator\Desktop>youtube-dl -v http://bangumi.bilibili.com/anime/2
1542/play#173288
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'http://bangumi.bilibili.com/anime/21542/play#
173288']
[debug] Encodings: locale cp936, fs mbcs, out cp936, pref cp936
[debug] youtube-dl version 2018.02.08
[debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: none
[debug] Proxy map: {}
[BiliBili] 173288: Downloading webpage
[BiliBili] Downloading episode 173288. To download all videos in anime 21542, re
-run youtube-dl with http://bangumi.bilibili.com/anime/21542
[BiliBili] 173288: Downloading JSON metadata
[BiliBili] 173288: Downloading video info page
ERROR: BiliBili said: video is hidden
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpn4uo9muj\bu
ild\youtube_dl\YoutubeDL.py", line 785, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpn4uo9muj\bu
ild\youtube_dl\extractor\common.py", line 440, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpn4uo9muj\bu
ild\youtube_dl\extractor\bilibili.py", line 131, in _real_extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpn4uo9muj\bu
ild\youtube_dl\extractor\bilibili.py", line 80, in _report_error
youtube_dl.utils.ExtractorError: BiliBili said: video is hidden

icve commented 6 years ago

you-get (another popular video download tool written in python) allows you to download a single bilibili bangumi episode.

screenshot_2018-09-27_13-52-41

run you-get <URL> to download a single episode.

if you need multiple episodes you could try you-get $(python -c 'print(" ".join("https://www.bilibili.com/bangumi/play/ep50511"[:-2]+str(i) for i in range(10, 36))))

This downloads ep5051110 to ep5051135, the command should works on a bash shell environment.

20NE commented 3 years ago

Still unfixed on youtube-dl 2020.12.07

D0LLYNH0 commented 3 years ago

Still unfixed on youtube-dl 2021.04.26

Oscar-Deng commented 3 years ago

Still unfixed on youtube-dl 2021.07.22

ytdl-org / youtube-dl