search

LonamiWebs / Telethon

Pure Python 3 MTProto API Telegram client library, for bots too!
https://docs.telethon.dev
MIT License
10.01k stars 1.41k forks source link

Cannot obtain Channel via get_entity by it's ID #4084

Open NotStatilko opened 1 year ago

NotStatilko commented 1 year ago

Checklist

Hi. Recently i bumped Telethon version from v1.25.4 to the most recent v1.28.2 and discovered that it's now impossible to obtain Channel entity via TelegramClient.get_entity. This behaviour started from the v1.28.0.

Maybe worth noting that i use a StringSession instead of default SQLite.

Code that causes the issue

...
# This doesn't work
await client.get_entity(channel_id)

# This doesn't work too
entity = await client.get_input_entity(channel_id)
await client.get_entity(entity)

# But this is OK
peer_channel = PeerChannel(channel_id)
channel_entity = await client.get_entity(entity)

Traceback

Traceback (most recent call last):
  File "/home/non/Scripts/py_projects/tgbox-env/tgbox/a_test.py", line 648, in <module>
    loop.run_until_complete(main())
  File "uvloop/loop.pyx", line 1501, in uvloop.loop.Loop.run_until_complete
  File "/home/non/Scripts/py_projects/tgbox-env/tgbox/a_test.py", line 587, in main
    print(await drb.tc.get_entity(ie))
  File "/home/non/Scripts/py_projects/tgbox-env/lib/python3.9/site-packages/telethon/client/users.py", line 332, in get_entity
    result.append(id_entity[utils.get_peer_id(x)])
KeyError: 1610806777

Is it OK?

Lonami commented 1 year ago

The traceback seems cut, as the error is missing. Could you provide the full error log?

v1.28 did introduce some changes to the cache of entities, but the traceback doesn't look right.

jw-star commented 1 year ago

I also have this problem

NotStatilko commented 1 year ago

Hi @Lonami. Sorry, i really forgot to include it xD

Here's full: image

With some quick test i discovered that this error related to the IDs with removed -100 prefixes

channel_id_with_minus = -100XXXXXXXXXX # Works fine
channel_id = XXXXXXXXXX # Raises KeyError

This error will not be raised if we firstly call get_entity on channel_id_with_minus and then channel_id, most probably because it goes to cache. This statement is true only for SQLite Session.

It seems that utils.get_peer_id doesn't add -100 prefix.

Lonami commented 1 year ago

That's the first time I'm seeing the above error. What does it show when you do the following?:

print(await client.get_input_entity(channel_id))
NotStatilko commented 1 year ago

That's the first time I'm seeing the above error. What does it show when you do the following?:

print(await client.get_input_entity(channel_id))

print(await client.get_input_entity(1610806777)) -> InputPeerUser(user_id=1610806777, access_hash=2328109287557450445)

print(await client.get_input_entity(-1001610806777)) -> InputPeerChannel(channel_id=1610806777, access_hash=2328109287557450445)

(By using StringSession with the latest v1 commits)

Lonami commented 1 year ago

For some reason, you seem to have the same ID in the cache with and without the mark. What does it show if you print(self.session._entities)?

Lonami commented 1 year ago

It might also be interesting to print the traceback and objects in https://github.com/LonamiWebs/Telethon/blob/07a7a8b4045dc60f733f8ebe5ced0bdf6ce721ac/telethon/sessions/memory.py#L150-L151, so that we know what is causing those to be saved that way:

    def process_entities(self, tlo):
        self._entities |= set(self._entities_to_rows(tlo))
        print('processing', tlo)
        import traceback
        for line in traceback.format_stack():
            print(line.strip())
        print('--')
NotStatilko commented 1 year ago

For some reason, you seem to have the same ID in the cache with and without the mark. What does it show if you print(self.session._entities)?

{(-1001610806777, 2328109287557450445, None, None, 'TGBOX[01]: v1.0.sql'), (894756050, 1341376998091936395, 'ndhsunqze1cexlhtgc1vg9tz3', '<...>', '/home/non/test/')}

'<...>' is my phone number. If i understand something, there is no duplicates.

When i made a little investigations further i found out that id_entity in get_entity store ID with mark, like this: {-1001610806777: <telethon.tl.types.Channel object at 0x7f0fd86396d0>}

And when we ask get_entity to give us Channel by the ID without -100 mark, it just can't find it in id_entity because is has only ID with mark.

The utils.get_peer_id doesn't add -100 mark to ID here (as i think should be expected?), so it's result in KeyError.

utils.get_peer_id just return ID as is if it's type is integer, but it should be prefixed with -100?

Based on my cursory research, it seems to me that the problem is in utils.get_peer_id. Maybe SQLite session store ID without -100 prefix, so it's presented in the id_entity (please note this). But StringSession seems to store only IDs with mark.

NotStatilko commented 1 year ago

When i made a little investigations further i found out that id_entity in get_entity store ID with mark, like this: {-1001610806777: <telethon.tl.types.Channel object at 0x7f0fd86396d0>}

Ok, now it seems that it's not true, id_entity is just empty dict when we call get_entity(1610806777), sorry :eyeroll

I will try to investigate it more later by myself.

Lonami commented 1 year ago

Nevermind, the above isn't gonna work.

NotStatilko commented 1 year ago

Nevermind, the above isn't gonna work.

Ok. So to receive channel we should specify ID with -100 prefix?

Lonami commented 1 year ago

The second commit linked should fix the issue.

NotStatilko commented 1 year ago

The second commit linked should fix the issue.

Missed your commit :o

I tested it and it seems that this commit doesn't change anything. Try this code:

from telethon.sync import TelegramClient

API_ID = ...
API_HASH = ...

# @telegram channel ID, need to be subscribed
TELEGRAM_CHANNEL_ID = 1005640892

with TelegramClient('_session', API_ID, API_HASH) as client:
    print(client.get_entity(TELEGRAM_CHANNEL_ID))

Result:

Traceback (most recent call last):
  File "/home/non/Scripts/py_projects/tgbox-env/tgbox/d_test.py", line 10, in <module>
    print(client.get_entity(TELEGRAM_CHANNEL_ID))
  File "/home/non/Scripts/py_projects/tgbox-env/lib/python3.9/site-packages/telethon/sync.py", line 39, in syncified
    return loop.run_until_complete(coro)
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/non/Scripts/py_projects/tgbox-env/lib/python3.9/site-packages/telethon/client/users.py", line 287, in get_entity
    inputs.append(await self.get_input_entity(x))
  File "/home/non/Scripts/py_projects/tgbox-env/lib/python3.9/site-packages/telethon/client/users.py", line 463, in get_input_entity
    raise ValueError(
ValueError: Could not find the input entity for PeerUser(user_id=1005640892) (PeerUser). Please read https://docs.telethon.dev/en/stable/concepts/entities.html to find out more details.

Getting Channel via ID without mark (1005640892) will work only if we firstly call get_entity(-1001005640892) (with mark), then [i think] library will pull out entity from the SQLite session file on get_entity(1005640892). This is the same situation that i described in my previous comment.

StringSession will fail (on ask without -100) even if we firstly ask for entity with marked ID.

NotStatilko commented 1 year ago

Getting Channel via ID without mark (1005640892) will work only if we firstly call get_entity(-1001005640892)

The below additional to the get_input_entity (code) will fix this

...
        # If we're a bot and the user has messaged us privately users.getUsers
        # will work with access_hash = 0. Similar for channels.getChannels.
        # If we're not a bot but the user is in our contacts, it seems to work
        # regardless. These are the only two special-cased requests.
        peer = utils.get_peer(peer)
        if isinstance(peer, types.PeerUser):
            users = await self(functions.users.GetUsersRequest([
                types.InputUser(peer.user_id, access_hash=0)]))

            if users and not isinstance(users[0], types.UserEmpty):
                # If the user passed a valid ID they expect to work for
                # channels but would be valid for users, we get UserEmpty.
                # Avoid returning the invalid empty input peer for that.
                #
                # We *could* try to guess if it's a channel first, and if
                # it's not, work as a chat and try to validate it through
                # another request, but that becomes too much work.
                return utils.get_input_peer(users[0])

            # = This block ============================ #
            try:
                channels = await self(functions.channels.GetChannelsRequest([
                    types.InputChannel(peer.user_id, access_hash=0)]))
                return utils.get_input_peer(channels.chats[0])
            except errors.ChannelInvalidError:
                pass
             # ======================================= #
...

However i don't know if this is valid fix because comment states that

We could try to guess if it's a channel first, and if it's not, work as a chat and try to validate it through another request, but that becomes too much work.

While this seems to make a fix to the SQLite session, the String one still doesn't work. The problem is that for some reason StringSession cache Channel ID without mark as InputPeerUser, so we receive this exception.

The first mentioned here exception (KeyError) disappears if we remove this block (isn't good for sure :) with the addition of upper one (try/except) and all works just as expected (we can retrieve Channel by ID without mark).

That's all that i found as for now.

Lonami commented 1 year ago 
ValueError: Could not find the input entity for PeerUser(user_id=1005640892) (PeerUser). Please read https://docs.telethon.dev/en/stable/concepts/entities.html to find out more details.

is a different error from

KeyError: 1610806777

which is what https://github.com/LonamiWebs/Telethon/commit/980f8b32fc16ff1ec6708b93fd1f923b8c684c3d fixed. Note how this "new" error is on line 332 whereas the old one is in 287 (i.e. it's hitting a different error at a later point).

While this seems to make a fix to the SQLite session, the String one still doesn't work. The problem is that for some reason StringSession cache Channel ID without mark as InputPeerUser, so we receive this exception.

That doesn't seem to be the case. In https://github.com/LonamiWebs/Telethon/issues/4084#issuecomment-1535304234 you showed that print(self.session._entities) gave (-1001610806777, 2328109287557450445, ..., so with the -100 mark.

The first mentioned here exception (KeyError) disappears if we remove this block (isn't good for sure :) with the addition of upper one (try/except) and all works just as expected (we can retrieve Channel by ID without mark).

Trying to fetch the user or channel with access_hash of 0 is more of a hacky last-resort attempt at getting something that works. It's not something I would rely on. (I really dislike the fact that it works, as it defeats the purpose of access-hashes in the first place if the server knows what we can access on its own and the lines are blurry.)

If you know it's a channel, use the -100ID form or PeerChannel(ID) and the library will try its best. I don't think there's more to fix here.

NotStatilko commented 1 year ago

That doesn't seem to be the case. In https://github.com/LonamiWebs/Telethon/issues/4084#issuecomment-1535304234 you showed that print(self.session._entities) gave (-1001610806777, 2328109287557450445, ..., so with the -100 mark.

Just to clarify (i don't think this is really important but maybe this will pop out later somewhere else):

from telethon.sync import TelegramClient
from telethon.sessions import StringSession

API_ID = ...
API_HASH = ...

SESSION = StringSession('...')

with TelegramClient(api_id=API_ID, api_hash=API_HASH, session=SESSION) as client:
    print(client.get_entity(-1001005640892))
    print(client.session.get_input_entity(1005640892))
    print(client.get_entity(1005640892))

Result in:

Channel(id=1005640892, title='Telegram News', photo=ChatPhoto(photo_id=4967709344246376726, dc_id=1, has_video=True, stripped_thumb=b'\x01\x08\x08\x7f\xc9\xe5\x9eN\xfa(\xa2\xbaN\x1b\x9f'), date=datetime.datetime(2023, 5, 5, 20, 57, 58, tzinfo=datetime.timezone.utc), creator=False, left=False, broadcast=True, verified=True, megagroup=False, restricted=False, signatures=False, min=False, scam=False, has_link=False, has_geo=False, slowmode_enabled=False, call_active=False, call_not_empty=False, fake=False, gigagroup=False, noforwards=False, join_to_send=False, join_request=False, forum=False, access_hash=7025306665282166182, username='telegram', restriction_reason=[], admin_rights=None, banned_rights=None, default_banned_rights=None, participants_count=None, usernames=[])

InputPeerUser(user_id=1005640892, access_hash=7025306665282166182)

Traceback (most recent call last):
  File "/home/non/Scripts/py_projects/tgbox-env/tgbox/d_test.py", line 12, in <module>
    print(client.get_entity(1005640892))
  File "/home/non/Scripts/py_projects/tgbox-env/lib/python3.9/site-packages/telethon/sync.py", line 39, in syncified
    return loop.run_until_complete(coro)
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/non/Scripts/py_projects/tgbox-env/lib/python3.9/site-packages/telethon/client/users.py", line 334, in get_entity
    result.append(id_entity[utils.get_peer_id(x, add_mark=False)])
KeyError: 1005640892

The same code, tag v1.27.0:

Channel(id=1005640892, title='Telegram News', photo=ChatPhoto(photo_id=4967709344246376726, dc_id=1, has_video=True, stripped_thumb=b'\x01\x08\x08\x7f\xc9\xe5\x9eN\xfa(\xa2\xbaN\x1b\x9f'), date=datetime.datetime(2023, 5, 5, 20, 57, 58, tzinfo=datetime.timezone.utc), creator=False, left=False, broadcast=True, verified=True, megagroup=False, restricted=False, signatures=False, min=False, scam=False, has_link=False, has_geo=False, slowmode_enabled=False, call_active=False, call_not_empty=False, fake=False, gigagroup=False, noforwards=False, join_to_send=False, join_request=False, forum=False, access_hash=7025306665282166182, username='telegram', restriction_reason=[], admin_rights=None, banned_rights=None, default_banned_rights=None, participants_count=None, usernames=[])

InputPeerUser(user_id=1005640892, access_hash=7025306665282166182)

Channel(id=1005640892, title='Telegram News', photo=ChatPhoto(photo_id=4967709344246376726, dc_id=1, has_video=True, stripped_thumb=b'\x01\x08\x08\x7f\xc9\xe5\x9eN\xfa(\xa2\xbaN\x1b\x9f'), date=datetime.datetime(2023, 5, 5, 20, 57, 58, tzinfo=datetime.timezone.utc), creator=False, left=False, broadcast=True, verified=True, megagroup=False, restricted=False, signatures=False, min=False, scam=False, has_link=False, has_geo=False, slowmode_enabled=False, call_active=False, call_not_empty=False, fake=False, gigagroup=False, noforwards=False, join_to_send=False, join_request=False, forum=False, access_hash=7025306665282166182, username='telegram', restriction_reason=[], admin_rights=None, banned_rights=None, default_banned_rights=None, participants_count=None, usernames=[])

The best way to operate with Channel ID is wrap it in PeerChannel. That's make sense. Thanks, Lonami!

Lonami commented 1 year ago

Yeah I don't understand how line 334 can fail if the dictionary is built with add_mark=False and accessed in the same manner. Maybe I'll revisit this at some point.

speauty commented 1 year ago

interesting....i use this api to get channel by its id...emmm, the result of some groups which i'm not in is "Could not find the input entity for PeerChannel(channel_id=1273185727) (PeerChannel)", and other groups which i'm in is ok:

# channel_id is real-id of channel
await tg_client.get_entity(types.PeerChannel(channel_id))
MelodyEars commented 11 months ago

In this code, I accessed the channel through another account that was already subscribed to the channel and received the object

Channel(id=2222222222, title='NameChannel', photo=ChatPhoto(photo_id=5884109877490597299, dc_id=4, has_video=False, stripped_thumb=b'\x01\x08\x08\xc9\xc2\xec\xce~l\xf4\xa2\x8a(z\x81'), date=datetime.datetime(2023, 12, 20, 9, 44, 2, tzinfo=datetime.timezone.utc), creator=False, left=False, broadcast=True, verified=False, megagroup=False, restricted=False, signatures=False, min=False, scam=False, has_link=False, has_geo=False, slowmode_enabled=False, call_active=False, call_not_empty=False, fake=False, gigagroup=False, noforwards=False, join_to_send=False, join_request=False, forum=False, stories_hidden=False, stories_hidden_min=False, stories_unavailable=True, access_hash=2332663674881728592, username=None, restriction_reason=[], admin_rights=None, banned_rights=None, default_banned_rights=None, participants_count=None, usernames=[], stories_max_id=None, color=PeerColor(color=9, background_emoji_id=5298529109570239315))

Then through except InviteHashExpiredError used the following function

invite = await client(ExportChatInviteRequest(peer=InputPeerChannel(access_hash=channel_obj.access_hash, channel_id=channel_obj.id)))

Error:

Traceback (most recent call last):

  File "home\PycharmProjects\tg_actions_with_group\scripts\test.py", line 221, in tst_main
    new_join_info = await client(ImportChatInviteRequest(hash=channel_url))
                          │      │                            └ 'WbDFfZ2OOvdjY2M0'
                          │      └ <class 'telethon.tl.functions.messages.ImportChatInviteRequest'>
                          └ <telethon.client.telegramclient.TelegramClient object at 0x000001EF9F13EE10>

  File "home\PycharmProjects\tg_actions_with_group\venv\Lib\site-packages\telethon\client\users.py", line 30, in __call__
    return await self._call(self._sender, request, ordered=ordered)
                 │    │     │    │        │                └ False
                 │    │     │    │        └ <telethon.tl.functions.messages.ImportChatInviteRequest object at 0x000001EF9D69B5D0>
                 │    │     │    └ <telethon.network.mtprotosender.MTProtoSender object at 0x000001EF9EE07F50>
                 │    │     └ <telethon.client.telegramclient.TelegramClient object at 0x000001EF9F13EE10>
                 │    └ <function UserMethods._call at 0x000001EFFF448FE0>
                 └ <telethon.client.telegramclient.TelegramClient object at 0x000001EF9F13EE10>
  File "home\PycharmProjects\tg_actions_with_group\venv\Lib\site-packages\telethon\client\users.py", line 87, in _call
    result = await future
                   └ <Future finished exception=InviteHashExpiredError('The chat the user tried to join has expired and is not valid anymore (caus...

telethon.errors.rpcerrorlist.InviteHashExpiredError: The chat the user tried to join has expired and is not valid anymore (caused by ImportChatInviteRequest)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "home\PycharmProjects\tg_actions_with_group\scripts\test.py", line 253, in <module>
    asyncio.run(tst_main())
    │       │   └ <function tst_main at 0x000001EF9DCB2660>
    │       └ <function run at 0x000001EFFD69A200>
    └ <module 'asyncio' from 'C:\\Users\\King\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\asyncio\\__init__.py'>

  File "home\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           │      │   └ <coroutine object tst_main at 0x000001EF9DCF1340>
           │      └ <function Runner.run at 0x000001EFFDB5DE40>
           └ <asyncio.runners.Runner object at 0x000001EFFB7DFE50>
  File "C:\Users\King\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           │    │     │                  └ <Task pending name='Task-1' coro=<tst_main() running at home\PycharmProjects\tg_actions_with_group\venv\Lib\site-pac...
           │    │     └ <function BaseEventLoop.run_until_complete at 0x000001EFFDB57920>
           │    └ <ProactorEventLoop running=True closed=False debug=False>
           └ <asyncio.runners.Runner object at 0x000001EFFB7DFE50>
  File "home\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 640, in run_until_complete
    self.run_forever()
    │    └ <function ProactorEventLoop.run_forever at 0x000001EFFDC27A60>
    └ <ProactorEventLoop running=True closed=False debug=False>
  File "home\AppData\Local\Programs\Python\Python311\Lib\asyncio\windows_events.py", line 321, in run_forever
    super().run_forever()
  File "home\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 607, in run_forever
    self._run_once()
    │    └ <function BaseEventLoop._run_once at 0x000001EFFDB5D6C0>
    └ <ProactorEventLoop running=True closed=False debug=False>
  File "home\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 1922, in _run_once
    handle._run()
    │      └ <function Handle._run at 0x000001EFFD69AD40>
    └ <Handle Task.task_wakeup(<Future finished result=None>)>
> File "home\AppData\Local\Programs\Python\Python311\Lib\asyncio\events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
    │    │            │    │           │    └ <member '_args' of 'Handle' objects>
    │    │            │    │           └ <Handle Task.task_wakeup(<Future finished result=None>)>
    │    │            │    └ <member '_callback' of 'Handle' objects>
    │    │            └ <Handle Task.task_wakeup(<Future finished result=None>)>
    │    └ <member '_context' of 'Handle' objects>
    └ <Handle Task.task_wakeup(<Future finished result=None>)>

  File "home\PycharmProjects\tg_actions_with_group\scripts\join_channel.py", line 231, in tst_main
    invite = await client(ExportChatInviteRequest(peer=InputPeerChannel(access_hash=channel_obj.access_hash, channel_id=channel_obj.id)))
                   │      │                            │                            │           │                       │           └ 2128475717
                   │      │                            │                            │           │                       └ <telethon.tl.types.Channel object at 0x000001EF9EE05FD0>
                   │      │                            │                            │           └ 2332663674881728592
                   │      │                            │                            └ <telethon.tl.types.Channel object at 0x000001EF9EE05FD0>
                   │      │                            └ <class 'telethon.tl.types.InputPeerChannel'>
                   │      └ <class 'telethon.tl.functions.messages.ExportChatInviteRequest'>
                   └ <telethon.client.telegramclient.TelegramClient object at 0x000001EF9F13EE10>

  File "home\PycharmProjects\tg_actions_with_group\venv\Lib\site-packages\telethon\client\users.py", line 87, in _call
    result = await future
                   └ <Future finished exception=ChannelInvalidError('Invalid channel object. Make sure to pass the right types, for instance makin...

telethon.errors.rpcerrorlist.ChannelInvalidError: Invalid channel object. Make sure to pass the right types, for instance making sure that the request is designed for channels or otherwise look for a different one more suited (caused by ExportChatInviteRequest)
Alnevis commented 6 months ago

interesting....i use this api to get channel by its id...emmm, the result of some groups which i'm not in is "Could not find the input entity for PeerChannel(channel_id=1273185727) (PeerChannel)", and other groups which i'm in is ok:

# channel_id is real-id of channel
await tg_client.get_entity(types.PeerChannel(channel_id))

is there any chance to get by ID the entity of channels or supergroups which you are not a member of?

alex123321-wq commented 3 months ago

Hi. I have a simular problem but with user_id, no with channel_id. Tried this - https://tl.telethon.dev/constructors/input_user.html

client = TelegramClient(session, api_id, api_hash)
client.connect()
...
# user_id = xxxxx (Random user_id from the chat where I'm an admin and that user is member of my chat)

entity = client.get_entity(InputUser(user_id=xxxxx, access_hash=yyyy))
Traceback (most recent call last):
  File "...../1.py", line 22, in <module>
    entity = client.get_entity(InputUser(user_id=xxxxx, access_hash=yyyy))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/telethon/sync.py", line 39, in syncified
    return loop.run_until_complete(coro)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/telethon/client/users.py", line 339, in get_entity
    result.append(id_entity[utils.get_peer_id(x, add_mark=False)])
                  ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: xxxxx (user_id provided earlier)

Python 3.12.4, Telethon 1.36.0