tankeryang / banciyuan-downloader

半次元coser图片批量下载,支持分类保存、断点续传、无重复下载、更新时只下载最新发布,不与本地重复
33 stars 2 forks source link

显示找不到路径 #5

Closed Darkari closed 5 years ago

Darkari commented 5 years ago

image 我开了管理员权限也会有这个问题....

tankeryang commented 5 years ago

@Darkari 你直接执行run.py会怎样呢?昨天我push了最新的代码上去,现在的是不需要登录名和密码的版本。你可以pull一份代码试试。我再看看是神马情况,估计是创建文件夹有问题

tankeryang commented 5 years ago

@Darkari 或者你试试下最新的 release exe文件执行看看o不ok,顺便帮我测试一下 @_@

Darkari commented 5 years ago

我跑了run.py 不行 后来我是改了一下代码,因为我是只跑某些作品而不是爬作者下面所有作品的 大概就是不需要按照coser名字新建文件夹,这样子就没问题 你说那个exe文件我吃个饭回来试试

话说因为我是只跑特定作品页面的,可不可以是按照第一个tag名新建文件夹(因为这个一般就是cos角色的作品名),然后每个作品文件夹的名字是coser名字+剩余的tag,这样子方便整理hhh 还有个就是每个图片的名称我下下来之后是按照1,2,3,4排序的,可不可以是能够照搬原来图片的名字呢 我也是新手很多地方没看懂

Darkari commented 5 years ago

@tankeryang 然后再运行的时候偶尔会有遇到下载某个作品到一半然后报错了 报错log大概是这样的:

`Traceback (most recent call last): File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 280, in recv_into return self.connection.recv_into(*args, **kwargs) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1715, in recv_into self._raise_ssl_error(self._ssl, result) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1521, in _raise_ssl_error raise WantReadError() OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 280, in recv_into return self.connection.recv_into(*args, **kwargs) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1715, in recv_into self._raise_ssl_error(self._ssl, result) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1521, in _raise_ssl_error raise WantReadError() OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 280, in recv_into return self.connection.recv_into(*args, **kwargs) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1715, in recv_into self._raise_ssl_error(self._ssl, result) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1521, in _raise_ssl_error raise WantReadError() OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 280, in recv_into return self.connection.recv_into(*args, **kwargs) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1715, in recv_into self._raise_ssl_error(self._ssl, result) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1521, in _raise_ssl_error raise WantReadError() OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 280, in recv_into return self.connection.recv_into(*args, **kwargs) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1715, in recv_into self._raise_ssl_error(self._ssl, result) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1521, in _raise_ssl_error raise WantReadError() OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\response.py", line 302, in _error_catcher yield File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\response.py", line 384, in read data = self._fp.read(amt) File "C:\Users\Gilbert\Anaconda3\lib\http\client.py", line 449, in read n = self.readinto(b) File "C:\Users\Gilbert\Anaconda3\lib\http\client.py", line 493, in readinto n = self.fp.readinto(b) File "C:\Users\Gilbert\Anaconda3\lib\socket.py", line 586, in readinto return self._sock.recv_into(b) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 296, in recv_into return self.recv_into(*args, kwargs) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 296, in recv_into return self.recv_into(*args, *kwargs) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 296, in recv_into return self.recv_into(args, kwargs) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 294, in recv_into raise timeout('The read operation timed out') socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\Gilbert\Anaconda3\lib\site-packages\requests\models.py", line 750, in generate for chunk in self.raw.stream(chunk_size, decode_content=True): File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\response.py", line 436, in stream data = self.read(amt=amt, decode_content=decode_content) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\response.py", line 401, in read raise IncompleteRead(self._fp_bytes_read, self.length_remaining) File "C:\Users\Gilbert\Anaconda3\lib\contextlib.py", line 99, in exit self.gen.throw(type, value, traceback) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\urllib3\response.py", line 307, in _error_catcher raise ReadTimeoutError(self._pool, None, 'Read timed out.') urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='img-bcy-qn.pstatp.com', port=443): Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "dl.py", line 47, in dl.get_pics() File "C:\Users\Gilbert\PycharmProjects\bcy\bcy_downloader.py", line 278, in get_pics pool.map(partial(self.get_pics, post_url, post_name), pics_url_list) File "C:\Users\Gilbert\Anaconda3\lib\multiprocessing\pool.py", line 266, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "C:\Users\Gilbert\Anaconda3\lib\multiprocessing\pool.py", line 644, in get raise self._value File "C:\Users\Gilbert\Anaconda3\lib\multiprocessing\pool.py", line 119, in worker result = (True, func(*args, *kwds)) File "C:\Users\Gilbert\Anaconda3\lib\multiprocessing\pool.py", line 44, in mapstar return list(map(args)) File "C:\Users\Gilbert\PycharmProjects\bcy\bcy_downloader.py", line 259, in __get_pics pic = self.session.get(pic_url.split('?')[0], timeout=3) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\requests\sessions.py", line 546, in get return self.request('GET', url, kwargs) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\requests\sessions.py", line 533, in request resp = self.send(prep, send_kwargs) File "C:\Users\Gilbert\Anaconda3\lib\site-packages\requests\sessions.py", line 686, in send r.content File "C:\Users\Gilbert\Anaconda3\lib\site-packages\requests\models.py", line 828, in content self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b'' File "C:\Users\Gilbert\Anaconda3\lib\site-packages\requests\models.py", line 757, in generate raise ConnectionError(e) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='img-bcy-qn.pstatp.com', port=443): Read timed out.`

tankeryang commented 5 years ago

@Darkari 很高兴你反馈的意见2333。关于win下新建coser文件夹的问题我今晚晚点再看看。然后就是分类的问题,因为考虑 通用性,我直接把 每一个作品当成一个文件夹,命名则是 作品标签拼接,图片 按编号保存。这样我也方便统计哪些作品是下载了,哪些没下。当然如果你有心的话可以改一下代码的逻辑,只取 第一个标签 作为文件夹名也是可以的。如果你要这么做的话,那么 图片的命名 也需要修改,因为有可能 好几个作品的图片都会放到一个文件夹下,按编号命名的方法就会发生冲突,建议直接改成 url 后几位这样来命名比较好。

tankeryang commented 5 years ago

@Darkari 报错的原因是网络问题了,可能半次元那边也有处理,发现某个ip请求过于频繁会加入黑名单之类的,一般都是正常的反爬措施。你可以找几个可用的ip作为代理池,然后换着爬,或者爬的过程中,每爬完一个作品就睡眠几秒然后继续之类的

tankeryang commented 5 years ago

@Darkari 那啥,我觉得应该是windows的文件夹路径 反人类的反斜杠问题 \。。你把路径都换成windows的\应该是可以的:E:\banciyuan这样

Darkari commented 5 years ago

@tankeryang 那个报错好像发生几率也不大,感觉应该是我自己网络的问题,因为有些图片挺大的下下来也要花点时间,这就类似与爬完休眠的感觉

tankeryang commented 5 years ago

嗯嗯,问题解决了,issue我这边就close掉了哦

Darkari commented 5 years ago

@tankeryang 好的,非常感谢大佬找到那个照片的url!自从前几天改了前段不能直接查看之后我还以为不能下原图了呢