[Feature Idea] Working with calls

tdlight-team / tdlight-telegram-bot-api

The TDLight Telegram Bot API is an actively enhanced fork of the original Bot API, featuring experimental user support, proxies, unlimited files size, and more.

https://t.me/TDLight

Boost Software License 1.0

128 stars 27 forks source link

[Feature Idea] Working with calls #35

Open spontanurlaub opened 3 years ago

spontanurlaub commented 3 years ago

It would be nice to work with calls using the bot api. Because streaming the data live would be quite complicated, I think it would be nice if you would be able to play audio (/video) files in a call and record the incoming audio (/video). That would allow a wide variety of interesting bots like a mailbox (if you don't answer the call within x seconds, it asks the user to leave a message that will then be sent into the chat as voice message), web-radio, voice assistant, alarm clocks with a real ringtone and much more

We would need to add libtgvoip to this project to support calls. Please leave your feedback.

andrew-ld commented 3 years ago

no libtgvoip should never be used programmatically outside the official clients because one wrong comma and everything crashes (in fact the dev doesn't work for telegram anymore), we should only implement the new library (https://github.com/TelegramMessenger/tgcalls)

spontanurlaub commented 3 years ago

Okay, I was not up to date with that, using tgcalls would be fine

giuseppeM99 commented 3 years ago

We would need to stream the audio/video content from and to the server, that would put a lot of stress on the botapi and a lot of work

spontanurlaub commented 3 years ago

It would be disabled by default of course and can only be activated with a command line comment. Allowing calls would let this api really stand out compared to most other userbot implementations

MarcoBuster commented 3 years ago

@code1mountain How would you handle the communication between the client and the (web) server?

andrew-ld commented 3 years ago

websocket?

spontanurlaub commented 3 years ago

I would not have any live communication from the server to the client. Just normal requests to play audio or start/stop/download the recording of the incoming audio/video stream.

JosXa commented 3 years ago

@code1mountain That is correct. AFAIK the only well-integrated and really usable implementations reside in MadelineProto and tdlib

luckydonald commented 3 years ago

Okey, maybe we should see what use cases we could have? Especially with those new group calls.

I can think of those:

Sending
- Internet Radio
- Sound boards
- Receiving
- Recorde as MP3
I think maybe a mp3 upload for sending and an internet-radio like stream for download could be sufficient for a first version? It wouldn't really support real time sending, but my thought would be that maybe with file_ids the playing could at least be timed properly enough?

Do we know what the resource usage could look like? I imagine that continuous streams would be a whole different deal for the server, as that'll mean more CPU, network activity and assigned ports and all that.

iiiiii1wepfj commented 3 years ago

@luckydonald noice

JosXa commented 3 years ago

Alarms (scheduled wake up calls)
High priority notifications
Get-me-outta-here calls (fake getting an important call by push of a button)

All those just need the ability to trigger a call, not to actually transmit audio.

luckydonald commented 3 years ago

We should also think about call events.

Currently that's just "text": "Voice Chat started" , and "text": "Voice Chat ended". Which I'd bet is an old tl-layer compatibility thing. Maybe that's why #36 or rather the upstream update to the newest version is taking a bit, and we get that for free then. Or they'll filter out those events. Let's see.

luckydonald commented 3 years ago

As video chat is a thing, too, probably RTMP could make sense. That's what big sites like Facebook, YouTube, Twitch and Periscope do for their live-streaming, so having something compatible with that would make integrations very easy.