CTransformers LLM

[acruikshank-wm]

Description

@vince-westmonroe has been playing around with locally hosted LLMs and LangChain. It turns out that LangChain has internal support for the CTransformers package that will download models from HuggingFaceHub and run them locally. This includes the quantized Llama-2 models that are SOTA for open source. It would be really nice to have this available in TestBench.

Acceptance

1. We have a new CTransformers LLM option in the LLM interface. This option allows you to specify an LLM model from huggingface hub and model type in addition to the normal hyperparameters.
2. We have the LLMSpec models on the front end and back end to configure, transport, save and instantiate CTransformers LangChain LLM within chains.
3. Documentation on how to use this option in the README (or additional documentation page). I believe the testbench server will need to be run outside the docker container for this to work.

[vince-westmonroe]

Interaction.tsx's runOnce function is hardwired to make an api call vs. having the option to interact with llm's via alternative methods (e.g., via ggm.bin file or via huggingface hub using ctransformers). An obvious way to conditionally interact with an llm is to provide the function with an llm type. In this scenario, the default interaction can be to call an openai api. If the llm type is huggingface hub, then it can call a new function that uses ctransformers.