search

NVIDIA / nim-deploy

A collection of YAML files, Helm Charts, Operator code, and guides to act as an example reference implementation for NVIDIA NIM deployment.
https://build.nvidia.com/
Apache License 2.0
141 stars 64 forks source link

Python test to try out function calling support #70

Closed vikalluru closed 1 week ago

vikalluru commented 3 months ago

Contains a python script to try out function calling support for LLAMA 3.1 NIM endpoints. It provides a sample tool call function, generates a tool call response, returns the tool call result based on the response and generates a new LLM response prompted with the tool call result. Useful to test multi-turn chatbot scenarios.

vikalluru commented 3 months ago

Can we move this file into it's own directory and add a very basic README.py that describes its usage/purpose?

Something like: docs/llm-nim-tooling-validation/*

or

scripts/llm-nim-function-calling-validation/*

Done.