LLMService Creation Changes

nv-morpheus / Morpheus

Morpheus SDK

Apache License 2.0

333 stars 124 forks source link

LLMService Creation Changes #1631

Closed shawn-davis closed 4 months ago

shawn-davis commented 4 months ago

Description

Adds the ability to instantiate LLM services/clients dynamically, so that models can be swapped out with command line arguments or config files instead of having to refactor pipeline code. Closes #1630

By Submitting this PR I confirm:

I am familiar with the Contributing Guidelines.
When the PR is ready for review, new or existing tests cover these changes.
When the PR is ready for review, the documentation is up to date with these changes.

copy-pr-bot[bot] commented 4 months ago

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

mdemoret-nv commented 4 months ago

/ok to test

mdemoret-nv commented 4 months ago

Merging to a feature branch to allow additional PRs to be created that fix the tests and documentation