运行错误

[yokowu]

lulu: [error] oops, something went wrong. lulu: don't panic, c'est la vie. please try the following steps: lulu: (1) Rule out any network problem. lulu: (2) Make sure lulu is up-to-date. lulu: (3) Check if the issue is already known, on lulu: https://github.com/iawia002/Lulu/issues lulu: (4) Run the command with '--debug' option, lulu: and report this issue with the full output.

[yokowu]

[DEBUG] url_locations: https://www.bilibili.com/video/av15658491/?from=search&seid=11921179244553020481 lulu: version 0.4.7, a tiny downloader that scrapes the web. lulu: Namespace(URL=['https://www.bilibili.com/video/av15658491/?from=search&seid=11921179244553020481'], cookies=None, debug=True, extractor_proxy=None, force=False, format=None, help=False, http_proxy=None, info=False, input_file=None, itag=None, json=False, no_caption=False, no_merge=False, no_proxy=False, output_dir='.', output_filename=None, password=None, player=None, playlist=False, socks_proxy=None, stream=None, thread=0, timeout=600, url=False, version=False) Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open encode_chunked=req.has_header('Transfer-encoding')) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send self.connect() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect server_hostname=server_hostname) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 407, in wrap_socket _context=self, _session=session) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 814, in init self.do_handshake() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1068, in do_handshake self._sslobj.do_handshake() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 689, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.6/bin/lulu", line 11, in sys.exit(main()) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/lulu/main.py", line 92, in main main(kwargs) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/lulu/common.py", line 1417, in main script_main(any_download, any_download_playlist, kwargs) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/lulu/common.py", line 1307, in script_main extra File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/lulu/common.py", line 1045, in download_main download(url, kwargs) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/lulu/common.py", line 1408, in any_download m.download(url, kwargs) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/lulu/extractor.py", line 78, in download_by_url self._prepare_download(kwargs) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/lulu/extractor.py", line 52, in _prepare_download self.prepare(*kwargs) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/lulu/extractors/bilibili.py", line 138, in prepare self.url = url_locations([self.url])[0] File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/lulu/common.py", line 432, in url_locations response = urlopen_with_retry(request.Request(url)) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/lulu/common.py", line 224, in urlopen_with_retry return request.urlopen(args, *kwargs) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen return opener.open(url, data, timeout) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open response = self._open(req, data) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open '_open', req) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain result = func(args) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open context=self._context, check_hostname=self._check_hostname) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>

[iawia002]

我复现不了，这看起来不是 lulu 的问题，你 python 版本是多少？

[meepuru]

E:\视频\bilibili\新建文件夹>lulu --debug https://www.bilibili.com/video/av18044828/ [DEBUG] url_locations: https://www.bilibili.com/video/av18044828/ lulu: version 0.4.7, a tiny downloader that scrapes the web. lulu: Namespace(URL=['https://www.bilibili.com/video/av18044828/'], cookies=None, debug=True, extractor_proxy=None, force=False, format=None, help=False, http_pro xy=None, info=False, input_file=None, itag=None, json=False, no_caption=False, no_merge=False, no_proxy=False, output_dir='.', output_filename=None, password=None , player=None, playlist=False, socks_proxy=None, stream=None, thread=0, timeout=600, url=False, version=False) Traceback (most recent call last): File "c:\program files\python36\lib\urllib\request.py", line 1318, in do_open encode_chunked=req.has_header('Transfer-encoding')) File "c:\program files\python36\lib\http\client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "c:\program files\python36\lib\http\client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "c:\program files\python36\lib\http\client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "c:\program files\python36\lib\http\client.py", line 1026, in _send_output self.send(msg) File "c:\program files\python36\lib\http\client.py", line 964, in send self.connect() File "c:\program files\python36\lib\http\client.py", line 1400, in connect server_hostname=server_hostname) File "c:\program files\python36\lib\ssl.py", line 407, in wrap_socket _context=self, _session=session) File "c:\program files\python36\lib\ssl.py", line 814, in init self.do_handshake() File "c:\program files\python36\lib\ssl.py", line 1068, in do_handshake self._sslobj.do_handshake() File "c:\program files\python36\lib\ssl.py", line 689, in do_handshake self._sslobj.do_handshake() socket.timeout: _ssl.c:761: The handshake operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "c:\program files\python36\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "c:\program files\python36\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Program Files\Python36\Scripts\lulu.exe__main.py", line 9, in File "c:\program files\python36\lib\site-packages\lulu\main__.py", line 92, in main main(kwargs) File "c:\program files\python36\lib\site-packages\lulu\common.py", line 1417, in main script_main(any_download, any_download_playlist, kwargs) File "c:\program files\python36\lib\site-packages\lulu\common.py", line 1307, in script_main extra File "c:\program files\python36\lib\site-packages\lulu\common.py", line 1045, in download_main download(url, kwargs) File "c:\program files\python36\lib\site-packages\lulu\common.py", line 1408, in any_download m.download(url, kwargs) File "c:\program files\python36\lib\site-packages\lulu\extractor.py", line 78, in download_by_url self._prepare_download(kwargs) File "c:\program files\python36\lib\site-packages\lulu\extractor.py", line 52, in _prepare_download self.prepare(*kwargs) File "c:\program files\python36\lib\site-packages\lulu\extractors\bilibili.py", line 138, in prepare self.url = url_locations([self.url])[0] File "c:\program files\python36\lib\site-packages\lulu\common.py", line 432, in url_locations response = urlopen_with_retry(request.Request(url)) File "c:\program files\python36\lib\site-packages\lulu\common.py", line 224, in urlopen_with_retry return request.urlopen(args, *kwargs) File "c:\program files\python36\lib\urllib\request.py", line 223, in urlopen return opener.open(url, data, timeout) File "c:\program files\python36\lib\urllib\request.py", line 526, in open response = self._open(req, data) File "c:\program files\python36\lib\urllib\request.py", line 544, in _open '_open', req) File "c:\program files\python36\lib\urllib\request.py", line 504, in _call_chain result = func(args) File "c:\program files\python36\lib\urllib\request.py", line 1361, in https_open context=self._context, check_hostname=self._check_hostname) File "c:\program files\python36\lib\urllib\request.py", line 1320, in do_open raise URLError(err) urllib.error.URLError:

E:\视频\bilibili\新建文件夹>

[iawia002]

@SGDIMO 你的 python 版本具体是多少？

[meepuru]

3.6.4 64位

[meepuru]

@iawia002

[iawia002]

@SGDIMO 我现在不太确定是什么问题，你下载每个视频都这样吗？

[meepuru]

@iawia002我尝试一下

[meepuru]

@iawia002 重装lulu过后貌似解决了...

[iawia002]

老是出现这种很奇怪的问题，不知道原因是什么

[iawia002]

@Yokong 你也重装一下 lulu 试一下呢

[Justsoos]

哦，上面看错了。 这个 _ssl.c:777 是常见错误，如果用 requests，设置 verify=False 就不常出了。

[Justsoos]

另外，第二个提问的，你怎么在别人问题底下说呢？都搞混了 http://www.bilibili.com/video/av18044828/ 你这个视频已经被删了，当然出错了 @SGDIMO

[yokowu]

@iawia002 重装后还是这个问题

[yokowu]

@Justsoos 怎么设置verify=False

[iawia002]

我确实想不通为什么你会出现这样的问题，B 站的证书也没有过期呀，你下每个视频都这样吗？

[iawia002]

你是只有 B 站的报错还是每个 https 的都报错

[Justsoos]

我也会有很多时候也出错，而且B站居多，就是这个 ssl.c 的各种连接错误，这是站方技术配置的问题。我没试过lulu 和you-get 怎么改底层，没时间啊。现在多用 requests，很方便。待会儿看看

[iawia002]

我晚上试着改一下发个版本你们再试试

[iawia002]

https://stackoverflow.com/questions/42098126/mac-osx-python-ssl-sslerror-ssl-certificate-verify-failed-certificate-verify

看这个能不能解决你们的问题

[Justsoos]

应该是这个 https://stackoverflow.com/questions/33770129/how-do-i-disable-the-ssl-check-in-python-3-x

[iawia002]

下 B 站的视频的话可以试一下 https://github.com/iawia002/annie go 语言写的，运行快很多

[Justsoos]

哈哈，annie，lulu，你这是 LOL 沉迷er来 github 抢注的啊。

[iawia002]

@Yokong 试一下运行这个脚本 /Applications/Python\ 3.6/Install\ Certificates.command 能不能解决你的问题，或者你用 brew 之类的安装 python 应该是不会出现这个问题的

tensorflow/tensorflow#10779

[Justsoos]

我用 win10 下安装的python3 试了下，现在十次里没有一次正常了，之前偶尔正常一次。但在 ubuntu 下还都正常。

λ lulu -d "https://www.bilibili.com/video/av15658491/?from=search&seid=11921179244553020481"
urllib.error.URLError: <urlopen error _ssl.c:761: The handshake operation timed out>

[yokowu]

优酷的, youtube的, B站的都不行

[Justsoos]

貌似是这个，用来更新 python ca证书： pip3 install -U certifi

[iawia002]

@Yokong /Applications/Python\ 3.6/Install\ Certificates.command 这个能不能解决你的问题

[yokowu]

@iawia002 试过了

[iawia002]

@Yokong 你改下代码，照着 #80 那样看行不行

[iawia002]

@Justsoos 你是说你 win10 下都不行？禁用 ssl 检查呢

[iawia002]

@Justsoos 能不能帮忙验证下 #80 那样改能不能解决问题

[yokowu]

@iawia002 我要怎么改?

[iawia002]

算了，我还是发个版本你再看能不能解决你的问题

[Justsoos]

对，我很少用win10 的lulu，刚才测了下，几乎没法用了。感觉来自you-get的古老代码已经落伍太多了，应该都换成 requests.Session()，这个库对 session 连接复用能力很强大 @iawia002 另外 go 版本 annie 的确速度快的惊人，但同样在B站出现类似错误： 'seid' 不是内部或外部命令，也不是可运行的程序 但很奇怪的是，annie 出这个错，还能正常运行完毕，python3 lulu 出这个错，无法运行了。

[iawia002]

@Justsoos annie 你是下载的二进制直接运行的吗？那些二进制都是我在 mac 上交叉编译的，可能其他系统有问题

[iawia002]

@Yokong 0.4.8

[Justsoos]

https://github.com/iawia002/annie/releases/download/0.1.1/annie_windows_amd64.exe 我下载的这个用的。估计是b站 api 的模板取的不完整，出现的 “seid” 错误，不是每次都出现这个错误。但 go 版本annie 的确很奇怪，每次都能运行正确。

[iawia002]

这个错看起来像是二进制文件的问题，我从没遇到过这个错，annie 本身也不会打印这个错误

[iawia002]

@Justsoos 如果你 win 上有 go 的话，用 go get 来安装应该就没问题了

[Justsoos]

哈哈，我还以为见鬼了，刚 pip3 upgrade 出来个 0.4.8 的lulu，原来这么快啊。

import ssl
ssl._create_default_https_context = ssl._create_unverified_context

我用的这个方式，禁止的 ssl 证书查询，但依然不行。

[Justsoos]

go get 遇上了伟大的墙，突然不好用了，就直接下载了二进制

[yokowu]

@iawia002 更新完,可以用了

[iawia002]

👌🏻 那我把 issue 关了，有其他问题再提

[yokowu]

@iawia002 感谢作者的付出!!! 向你学习

[Justsoos]

可能找到问题所在了，老旧的 urllib 不支持persistent connections，lulu 和 you-get parse url 的过程里，每个页面都要重新连接一次，多次频繁握手，不仅增加了 ssl 握手出错的机会，还可能让 B 站配置有问题的web服务器，或者可能B站配置了反ddos 机制起作用 -- 导致出错。

可以改一个支持 persistent connections 和 connection pool 的 urllib3 的 requests 的common.py，全面代替 urllib，应该就没问题了。 需要改的地方相当多啊。 https://github.com/iawia002/Lulu/issues/84

[Justsoos]

我突击改了一个 lulu -du xxx 全部用 requests 的 common.py ，替代了 urllib 测试，很多次测 B 站各链接，再也没有出现 ssl 错误。包括原来一直在 win10 上出错的。

[iawia002]

@Justsoos 我其实前几天改过一个版本 https://github.com/iawia002/Lulu/tree/requests

有时间我再完善一下