search

y-young / nazurin

🎉 Images collection done right. Telegram 图片收藏机器人 / A Telegram bot that helps you collect ACG illustrations.
http://nazurin.readthedocs.io
MIT License
260 stars 58 forks source link

TimeoutError while fetching Pixiv images #62

Closed Misaka13514 closed 1 year ago

Misaka13514 commented 1 year ago

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. Send a pixiv link to bot https://www.pixiv.net/artworks/104233490
  2. See error Error: Timeout, please try again.

Expected behavior Fetch the images and send them to storages.

Error output / Logs

2023-01-06 15:13:11,215 - nazurin.__init__ - INFO - Loaded 12 sites
2023-01-06 15:13:12,492 - nazurin.__init__ - INFO - Loaded 2 storage(s), using: ['Telegram', 'GoogleDrive']
2023-01-06 15:13:12,498 - nazurin.dispatcher - INFO - Set webhook
2023-01-06 15:13:13,311 - aiogram.executor - INFO - Bot: ナズーリン [@botname]
======== Running on http://0.0.0.0:80 ========
(Press CTRL+C to quit)
2023-01-07 05:47:37,261 - nazurin.bot - INFO - Collection update: site=pixiv, match=('104233490',)
2023-01-07 05:47:37,373 - aiohttp.access.web_log - INFO - 172.18.0.1 [07/Jan/2023:05:47:37 +0000] "POST /<TELEGRAM_BOT_TOKEN> HTTP/1.0" 200 152 "-" "-"
2023-01-07 05:47:37,656 - nazurin.api - INFO - Bookmarked artwork 104233490, privacy = public
2023-01-07 05:47:40,815 - nazurin.image - INFO - Got image size: 2150086
2023-01-07 05:47:48,990 - tenacity.decorators - INFO - TimeoutError() during nazurin.models.file.File.download execution, 1 of 5 attempted.
2023-01-07 05:48:00,993 - tenacity.decorators - INFO - TimeoutError() during nazurin.models.file.File.download execution, 2 of 5 attempted.
2023-01-07 05:48:13,989 - tenacity.decorators - INFO - TimeoutError() during nazurin.models.file.File.download execution, 3 of 5 attempted.
2023-01-07 05:48:28,990 - tenacity.decorators - INFO - TimeoutError() during nazurin.models.file.File.download execution, 4 of 5 attempted.
2023-01-07 05:48:47,991 - tenacity.decorators - INFO - TimeoutError() during nazurin.models.file.File.download execution, 5 of 5 attempted.
Traceback (most recent call last):
  File "/app/nazurin/__main__.py", line 56, in on_error
    raise exception
  File "/usr/local/lib/python3.8/site-packages/aiogram/dispatcher/dispatcher.py", line 1394, in process_response
    response = task.result()
  File "/app/nazurin/dispatcher.py", line 59, in update_collection
    await self.bot.update_collection(urls, message)
  File "/app/nazurin/bot.py", line 131, in update_collection
    await asyncio.gather(save, download)
  File "/app/nazurin/sites/pixiv/models.py", line 21, in download
    await super().download(headers=HEADERS, **kwargs)
  File "/app/nazurin/models/illust.py", line 35, in download
    await asyncio.gather(*tasks)
  File "/usr/local/lib/python3.8/site-packages/tenacity/_asyncio.py", line 48, in call
    do = self.iter(retry_state=retry_state)
  File "/usr/local/lib/python3.8/site-packages/tenacity/__init__.py", line 332, in iter
    raise retry_exc.reraise()
  File "/usr/local/lib/python3.8/site-packages/tenacity/__init__.py", line 168, in reraise
    raise self.last_attempt.result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.8/site-packages/tenacity/_asyncio.py", line 51, in call
    result = yield from fn(*args, **kwargs)
  File "/app/nazurin/models/file.py", line 46, in download
    await f.write(await response.read())
  File "/usr/local/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 1037, in read
    self._body = await self.content.read()
  File "/usr/local/lib/python3.8/site-packages/aiohttp/streams.py", line 375, in read
    block = await self.readany()
  File "/usr/local/lib/python3.8/site-packages/aiohttp/streams.py", line 397, in readany
    await self._wait("readany")
  File "/usr/local/lib/python3.8/site-packages/aiohttp/streams.py", line 304, in _wait
    await waiter
  File "/usr/local/lib/python3.8/site-packages/aiohttp/helpers.py", line 720, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError

Environment (please complete the following information):

Additional context I have tried to refresh Pixiv token by #9, does not work.

y-young commented 1 year ago

Seems working on my side.

What about other Pixiv links? Have you tried /pixiv_download? You may also try to download the image via the pximg.net link in command line (e.g. using wget or curl) on your server to see if it's working.

Misaka13514 commented 1 year ago

Have you tried /pixiv_download?

Error: Timeout, please try again.

You may also try to download the image via the pximg.net link in command line (e.g. using wget or curl) on your server to see if it's working.

$ curl -L 'https://i.pximg.net/img-original/img/2023/01/04/19/34/29/104233490_p0.png' \
                               -H 'authority: i.pximg.net' \
                               -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
                               -H 'accept-language: ja-JP,ja;q=0.9,en-US;q=0.8,en;q=0.7,zh-CN;q=0.6,zh;q=0.5' \
                               -H 'cache-control: no-cache' \
                               -H 'dnt: 1' \
                               -H 'pragma: no-cache' \
                               -H 'referer: https://www.pixiv.net/' \
                               -H 'sec-ch-ua: "Not?A_Brand";v="8", "Chromium";v="108"' \
                               -H 'sec-ch-ua-mobile: ?0' \
                               -H 'sec-ch-ua-platform: "Linux"' \
                               -H 'sec-fetch-dest: document' \
                               -H 'sec-fetch-mode: navigate' \
                               -H 'sec-fetch-site: cross-site' \
                               -H 'sec-fetch-user: ?1' \
                               -H 'upgrade-insecure-requests: 1' \
                               -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36' \
                               --compressed --output 104233490_p0.png
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 2099k  100 2099k    0     0  27797      0  0:01:17  0:01:17 --:--:-- 34576

Seems it is the issue of my network. (Time Total 01:17 for 2099k) I don't know why(

Misaka13514 commented 1 year ago

Modified TIMEOUT in env, it works now. Thank you.