search

LonamiWebs / Telethon

Pure Python 3 MTProto API Telegram client library, for bots too!
https://docs.telethon.dev
MIT License
10.01k stars 1.41k forks source link

"Cannot send requests while disconnected" error when trying to download all media from Channel #958

Closed dmitrdv closed 3 years ago

dmitrdv commented 6 years ago

I try to write code to download all messages and media from specific channel. Getting messages without downloading media works fine. But with file download it is failed usually earlier than on 10 iteration. I use Python 3.6, Telethon 1.2.

My code:

import telethon

# change on your api id and hash
api_id = 123
api_hash = 'some_api_hash'

client = telethon.TelegramClient('anon', api_id, api_hash)

async def run():  
    await client.connect()
    channel = await client.get_entity('varlamov')

    i = 0
    async for message in client.iter_messages(channel):
        i = i + 1
        print(str(i))

        await client.download_media(message, file='download/')

loop = asyncio.get_event_loop()
loop.run_until_complete(run())

Error:


future: <Task finished coro=<MTProtoSender._recv_loop() done, defined at /usr/local/lib/python3.6/dist-packages/telethon/network/mtprotosender.py:415> exception=CancelledError()>
concurrent.futures._base.CancelledError
Traceback (most recent call last):
  File "app.py", line 25, in <module>
    loop.run_until_complete(run())
  File "/usr/lib/python3.6/asyncio/base_events.py", line 468, in run_until_complete
    return future.result()
  File "app.py", line 20, in run
    await client.download_media(message, file='download/')
  File "/usr/local/lib/python3.6/dist-packages/telethon/client/downloads.py", line 142, in download_media
    media, file, date, progress_callback
  File "/usr/local/lib/python3.6/dist-packages/telethon/client/downloads.py", line 311, in _download_photo
    progress_callback=progress_callback)
  File "/usr/local/lib/python3.6/dist-packages/telethon/client/downloads.py", line 244, in download_file
    input_location, offset, part_size
  File "/usr/local/lib/python3.6/dist-packages/telethon/network/mtprotosender.py", line 202, in send
    raise ConnectionError('Cannot send requests while disconnected')
ConnectionError: Cannot send requests while disconnected```

Also I am trying to catch exception to check whether client is connected or not. And method is_connected() return True.
Lonami commented 6 years ago

Could you enable logging.INFO and tell me if there are "Exporting authorization for data center" messages?

dmitrdv commented 6 years ago

This log is for successful execution of "client.download_media(message, file='download/')":

INFO:Connecting to 149.154.167.91:443...
DEBUG:Connection attempt 1...
DEBUG:Connection success!
DEBUG:Starting send loop
DEBUG:Starting receive loop
INFO:Connection to 149.154.167.91 complete!
INFO:Downloading file in chunks of 131072 bytes
DEBUG:Packing 1 outgoing message(s) GetFileRequest...
DEBUG:Sending 104 bytes...
DEBUG:Receiving items from the network...
DEBUG:Outgoing messages 6591454118757691448 sent!
DEBUG:Decoding packet of 168 bytes...
DEBUG:Handling container
DEBUG:Handling RPC result for message 6591454106731421648
INFO:Received response without parent request: b'\xd5\x18j\t\x0e\xfe~\x00p\xbbu[\x00\x00\x00\x00\x01\x9c\xb3;\x99\x8cy[\x08\x00\x00\x00\x14\x00\x00\x00Y\xb4\xd6b\x15\xc4\xb5\x1c\x01\x00\x00\x008p\xd6\x15\x99\x8cy[~~d\x15~\r\xcc\xcdM\xf3w\x97\xe0\xba\x87~P\xa9\xb3e\xd3\x1a&\x87'
DEBUG:Handling acknowledge for [6591454118757691448]
DEBUG:Receiving items from the network...
DEBUG:Decoding packet of 121336 bytes...
DEBUG:Handling RPC result for message 6591454118757691448
DEBUG:Receiving items from the network...
DEBUG:Saving 121228 more bytes
DEBUG:Packing 1 outgoing message(s) GetFileRequest...
DEBUG:Sending 104 bytes...
DEBUG:Outgoing messages 6591454124035977072 sent!
DEBUG:Packing 1 outgoing message(s) MsgsAck...
DEBUG:Sending 120 bytes...
DEBUG:Outgoing messages 6591454124040634816 sent!
DEBUG:Decoding packet of 104 bytes...
DEBUG:Handling RPC result for message 6591454124035977072
DEBUG:Receiving items from the network...
INFO:Disconnecting borrowed sender for DC 4
INFO:Disconnecting from 149.154.167.91...
DEBUG:Closing current connection...
DEBUG:Cancelling 0 pending message(s)...
DEBUG:Cancelling the send loop...
DEBUG:Cancelling the receive loop...
INFO:Disconnection from 149.154.167.91 complete!

This log is for failed execution of "client.download_media(message, file='download/')":

INFO:Connecting to 149.154.167.91:443...
DEBUG:Connection attempt 1...
INFO:Disconnecting from 149.154.167.91...
DEBUG:Closing current connection...
DEBUG:Cancelling 0 pending message(s)...
DEBUG:Cancelling the send loop...
DEBUG:Cancelling the receive loop...
INFO:Disconnection from 149.154.167.91 complete!
WARNING:Attempt 1 at connecting failed: ConnectionResetError:
DEBUG:Connection attempt 2...
DEBUG:Connection success!
DEBUG:Starting send loop
DEBUG:Starting receive loop
ERROR:Task exception was never retrieved
future:  exception=CancelledError()>
concurrent.futures._base.CancelledError
INFO:Connection to 149.154.167.91 complete!
INFO:Downloading file in chunks of 131072 bytes
INFO:Disconnecting borrowed sender for DC 4
INFO:User is already disconnected!
Traceback (most recent call last):
  File "app.py", line 28, in 
    loop.run_until_complete(run())
  File "/usr/lib/python3.6/asyncio/base_events.py", line 468, in run_until_complete
    return future.result()
  File "app.py", line 23, in run
    await client.download_media(message, file='download/')
  File "/usr/local/lib/python3.6/dist-packages/telethon/client/downloads.py", line 142, in download_media
    media, file, date, progress_callback
  File "/usr/local/lib/python3.6/dist-packages/telethon/client/downloads.py", line 311, in _download_photo
    progress_callback=progress_callback)
  File "/usr/local/lib/python3.6/dist-packages/telethon/client/downloads.py", line 244, in download_file
    input_location, offset, part_size
  File "/usr/local/lib/python3.6/dist-packages/telethon/network/mtprotosender.py", line 202, in send
    raise ConnectionError('Cannot send requests while disconnected')
ConnectionError: Cannot send requests while disconnected
Lonami commented 6 years ago

INFO:Received response without parent request

That is #658 once again, in this case File(type=FileJpeg(), mtime=1534442352, bytes=b'').

Anyway, back to your issue, it's clear to see that in the first case the first attempt works, while in the second case, it fails (yet connects in the second attempt).

Lonami commented 6 years ago

I fail to understand how your log connects and disconnects when the code flow doesn't go like that.

Lonami commented 6 years ago

Seems to have fixed it.

ingria commented 4 years ago

I'm experiencing the same issue under similar circumstances: "Cannot send requests while disconnected" when mass-downloading media from channels. 

ConnectionError: Cannot send requests while disconnected
  File "app/controllers.py", line 117, in utils_getFile
    file = self.client.download_file(input_location, dc_id=dc_id)
  File "telethon/sync.py", line 39, in syncified
    return loop.run_until_complete(coro)
  File "asyncio/base_events.py", line 484, in run_until_complete
    return future.result()
  File "telethon/client/downloads.py", line 436, in download_file
    input_location, request_size=part_size, dc_id=dc_id):
  File "telethon/requestiter.py", line 75, in __anext__
    if await self._load_next_chunk():
  File "telethon/client/downloads.py", line 60, in _load_next_chunk
    cur = await self._request()
  File "telethon/client/downloads.py", line 70, in _request
    result = await self._sender.send(self.request)
  File "telethon/network/mtprotosender.py", line 170, in send
    raise ConnectionError('Cannot send requests while disconnected')

Also, I'm sometimes seeing this error (not sure if they are connected, but still):

AttributeError: 'NoneType' object has no attribute 'connect'
  File "telethon/network/mtprotosender.py", line 363, in _reconnect
    await self._connect()
  File "telethon/network/mtprotosender.py", line 224, in _connect
    connected = await self._try_connect(attempt)
  File "telethon/network/mtprotosender.py", line 273, in _try_connect
    await self._connection.connect(timeout=self._connect_timeout)
Lonami commented 4 years ago

No idea how I'm going to debug though.

ingria commented 4 years ago

Also this sequence of events seems to happen quite often:

04:43:08 - telethon.network.mtprotosender - Disconnecting from 149.154.167.91:443/TcpFull...
04:43:08 - telethon.network.mtprotosender - Disconnection from 149.154.167.91:443/TcpFull complete!
04:43:08 - telethon.network.mtprotosender - Failed reconnection attempt 3 with ConnectionError
04:46:21 - rpc_call - utils.getFile
04:46:21 - telethon.network.mtprotosender - Connecting to None...
04:46:21 - telethon.network.mtprotosender - Unexpected exception reconnecting on attempt 4
04:46:21 - rpc_call - utils.getFile
04:46:21 - exception - ConnectionError: Cannot send requests while disconnected

utils.getFile is my app method, its job is to ether call client.download_file, or get channel post (channels.GetMessagesRequest), extract file_reference and then call client.download_file.

byehack commented 4 years ago

same here on version 1.16.2 any way to fix it?

dumpload commented 4 years ago

is sync run account, and banned. but sync is connect. not disconnect

Lonami commented 4 years ago

same here on version 1.16.2

Could you post the full traceback with the error under the latest version (pip3 install -U https://github.com/LonamiWebs/Telethon/archive/master.zip --user)? Having a traceback where the lines match with the code is very valuable and would help me introduce bogus errors to see if I can get the same behaviour.

4vadim4 commented 3 years ago

@dmitrdv with Telethon==1.19.5 everything works well
try code bellow:

import telethon

api_id = <your id>
api_hash = '<your hash>'

client = telethon.TelegramClient('anon', api_id, api_hash)

async def run():
    await client.connect()
    channel = await client.get_entity('varlamov')

    i = 0
    async for message in client.iter_messages(channel):
        i = i + 1
        print(str(i))

        await client.download_media(message, file='download/')

client.loop.run_until_complete(run())
graysuit commented 3 years ago

instead define/connect client outside def and call def.


import telethon
def run():
    channel = await client.get_entity('varlamov')
    i = 0
    for message in client.iter_messages(channel):
        i = i + 1
        print(str(i))
        await client.download_media(message, file='download/')

api_id = <your id>
api_hash = '<your hash>'
client = telethon.TelegramClient('anon', api_id, api_hash)
client.connect()
run()
Lonami commented 3 years ago

Closing as this was reported a fairly long time ago and the cause is probably some connection issue.

Lonami commented 2 years ago

@Mrtznx markdown_it is not used by Telethon v1. Make sure to uninstall any current telethon you have, and then install the latest stable version from PyPi.