nv-morpheus / Morpheus

Morpheus SDK
Apache License 2.0
333 stars 124 forks source link

[FEA]: Update LLMService to allow dynamic instantiation #1630

Open shawn-davis opened 4 months ago

shawn-davis commented 4 months ago

Is this a new feature, an improvement, or a change to existing functionality?

Change

How would you describe the priority of this feature request

Medium

Please provide a clear description of problem this feature solves

I want to be able to instantiate LLM services/clients dynamically, so that models can be swapped out with command line arguments or config files instead of having to refactor pipeline code.

Describe your ideal solution

llm_service = LLMService.create(service, service_kwargs) llm_client = llm_service.get_client(model_kwargs)

Additional context

No response

Code of Conduct