search

dusty-nv / NanoLLM

Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.
https://dusty-nv.github.io/NanoLLM/
MIT License
196 stars 31 forks source link

system prompt file parsing #40

Closed bigrobinson closed 2 months ago

bigrobinson commented 2 months ago

I'm exploring function calling with Hermes-2-Pro-Llama-3-8B and following the recent research group video demonstration using agent studio plugin interface. I am wondering how the prompt text file (/data/prompts/nous_system.txt) is parsed into the system prompt. I see there some variables (TOOL_DOCS, GEOLOCATION, DATE) dereferenced in the file, but I can't find where the magic happens to substitute these values. The tools are not being populated. The location is correct. The month and day of the week are correct, but the date is off. Can you clarify this? Thank you!

Screenshot from 2024-08-24 07-42-58

bigrobinson commented 2 months ago

Update: I guess the variables names are case lowered and used to call tool functions. The issue with the date is just a typo in the Clock.time method (%-m to %-d to get day and not month). In the location tool, the geolocation method needs 'self' in the signature to pass inspect and load in agent studio. I see where tool_docs is under construction, and that's an easy cut and paste. All working fine now. Peace.