Forward Audio -> Transcribe With Whisper -> Create Tasks

0x4007 commented 11 months ago

Telegram allows group voice/video calls to be recorded.
When the call is finished, Telegram saves the recording in "saved messages"
It is VERY fast to forward that content to the bot.
The bot should automatically create a transcription using the OpenAI Whisper API.
Once the transcription is complete, the bot should interpret the transcript in a similar way to how it currently parses Telegram group chat messages, and create separate tasks.
We should be able to set a special "meetings" repository (configurable) that the bot will automatically file all the issues to.

Remarks

Technically this means that if you send a direct voice message to the bot, it should do the same thing.
In the future, we can experiment with confirming the repository that the user wants to file the issues to right after they forward the media. That way, if the call was related to research or business-development then we can file them all in the correct repo.
Whisper can not tell the difference between each speaker, but I've seen open source projects that do handle this. It would be interesting to identify the speaker when creating the issue.
When the issue is posted, it would be epic to trim and post the audio (and its transcription) as the issue description.

Tooling

Use diarization https://twitter.com/reach_vb/status/1729251580371689821

0x4007 commented 8 months ago

/start

ubiquibot[bot] commented 8 months ago



Warning!  This task was created over 91 days ago. Please confirm that this issue specification is accurate before starting.
Deadline Fri, Dec 22, 12:37 PM UTC

Registered Wallet
0x4007CE2083c7F3E18097aeB3A39bb8eC149a341d

Tips:

Use /wallet 0x0000...0000 if you want to update your registered payment wallet address.
Be sure to open a draft pull request as soon as possible to communicate updates on your progress.
Be sure to provide timely updates to us when requested, or you will be automatically unassigned from the task.

0x4007 commented 8 months ago

/wallet ubq.eth

ubiquibot[bot] commented 8 months ago

+ Successfully registered wallet address

0x4007 commented 8 months ago

/query @pavlovcik

ubiquibot[bot] commented 8 months ago

Property	Value
Wallet	0xefC0e701A824943b469a694aC564Aa1efF7Ab7dd

0x4007 commented 8 months ago

/wallet pavlovcik.eth

ubiquibot[bot] commented 8 months ago

+ Successfully registered wallet address

0x4007 commented 8 months ago

this is a test comment about apples

ubiquibot[bot] commented 8 months ago

! action has an uncaught error

PhantomCracker commented 7 months ago

@seprintour can you please help on connecting to the bot? I see that you are making the configurations file but I have some issues with that. I receive the following error message when I am running key setup:

Or do you think that you can help @pavlovcik ?

ubiquity / ubiquibot-telegram