Raptor123471 / DingoLingo

A Discord music bot written in Python with support for Youtube, SoundCloud, Spotify, Bandcamp, Twitter, and custom files.
GNU General Public License v3.0
282 stars 165 forks source link

Slow sync func #85

Open lk2322 opened 2 years ago

lk2322 commented 2 years ago

This func block the entire thread for a couple of seconds. Maybe run it in loop.run_in_executor? https://github.com/Raptor123471/DingoLingo/blob/270e750c47de5d2dd670b7f9bf02e3b903874911/musicbot/audiocontroller.py#L261

lk2322 commented 2 years ago

We can use decorator like this

from functools import wraps, partial
import asyncio

def run_async(func):
    @wraps(func)
    async def run(*args, loop=None, executor=None, **kwargs):
        if loop is None:
            loop = asyncio.get_event_loop()
        pfunc = partial(func, *args, **kwargs)
        return await loop.run_in_executor(executor, pfunc)

    return run
Raptor123471 commented 2 years ago

Can you provide an example? I have tried using a similar method but it does not work.

lk2322 commented 2 years ago

If I didn't forget something, I just used this decorator and added await before search_youtube

malikmajai commented 2 years ago

Did we found a solution to this issue that we can apply?

malikmajai commented 2 years ago

If I didn't forget something, I just used this decorator and added await before search_youtube

?

RafaelSolVargas commented 2 years ago

If I didn't forget something, I just used this decorator and added await before search_youtube

Well, i tried to do that, the main problem in that it's not that simple. To be more specific, the preload function that uses asyncio to run multiple tasks at once doesn't support running coroutine or awaitable functions, and as you can see, the search_youtube funct is within the func down, forcing the def down to be a coroutine (so that he can use await search_youtube).

https://github.com/Raptor123471/DingoLingo/blob/6790bbbb1823a0c88710d776ea7a9bfbd4f82fdc/musicbot/audiocontroller.py#L237-L245 This information was collected in this stackoverflow: https://stackoverflow.com/questions/46074841/why-coroutines-cannot-be-used-with-run-in-executor I tried creating two search_youtube functions, one sync and one async, but other bugs were showing up and crashing the bot, so I gave up.

The solution to execute multiple coroutine functions at once is also in the stackoverflow, it's the second answer, but it's not simple and will require some major changes in the logic that interact with YoutubeDL.

https://github.com/RafaelSolVargas/Vulkan/blob/2dbc6c3984b2024ee14502b8b8cd8fd06c967f75/Music/Downloader.py#L108-L124 This is how another discord music bot deal with that exactly problem, the key here is to have an organized way to work with YoutubeDL, possibly using a class