livekit / agents

Build real-time multimodal AI applications 🤖🎙️📹
https://docs.livekit.io/agents
Apache License 2.0
4.07k stars 429 forks source link

Cannot pass in custom start_arguments with cli.run_app(opts) #950

Open btakita opened 1 month ago

btakita commented 1 month ago

I want to use pytorch's share_memory() & pass in models + Tensors into the worker processes from the main process. Unfortunately, it does not look like there is a way to do that from cli.run_apps. I have to duplicate the entire call stack which includes:

Just to add a hook to access proc.start_arguments. Please add a hook into your library so I can pass in custom start_arguments/_user_args.

Thank you

jezell commented 1 month ago

We just instantiate the pipeline agent directly and did our own job scheduling. It is a little bit of work to figure it out, but might be worth it if you want to really have a lot more control. You don't need the Worker / ProcPool / ProcJobExecutor or cli modules to go that route.