search

NVIDIA / NeMo-Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
Other
3.97k stars 363 forks source link

Specific LLM for each task #163

Closed austinmw closed 10 months ago

austinmw commented 10 months ago

Hi, is it possible to specify specific LLMs for each task? For example I'd like to use Claude v2 for generate_bot_message and Claude Instant for all other tasks.

austinmw commented 10 months ago

Sorry, found answer, seems like I can just do this:

models:
  - type: main
    engine: amazon_bedrock
    model: anthropic
    parameters:
      model_id: anthropic.claude-instant-v1
      model_kwargs:
        max_tokens_to_sample: 1000
        temperature: 0.0
        stop_sequences: ["\n\nHuman:", "\nUser:"]

  - type: generate_bot_message
    engine: amazon_bedrock
    model: anthropic
    parameters:
      model_id: anthropic.claude-v2
      model_kwargs:
        max_tokens_to_sample: 10000
        temperature: 0.3
        stop_sequences: ["\n\nHuman:", "\nUser:"]