First of all, Nous hermes 2 pro is a beast of an 7B reasoning model.
I've been experimenting with the Hermes 2 Pro, and I've noticed some maybe interesting behavior.
According to the repository instructions, I initially used the model with \\ and , etc with converted functions to openai etc...
However, I discovered that it performs significantly better (at least for my use cases) when directly interacting with raw API calls (no tools defined) and no langchain.
Specifically, I set up each API endpoint in an OpenAPI specification and provided it in user part of the prompt, allowing the model to dynamically choose any endpoint it deems useful, while disregarding those it doesn't need.
It was able to use tools when needed, and avoid tools altogether when not needed even in long chat sessions.
Example "Tools" used: generate_image, search_vector_store, expand_image, etc...
Sample that Hermes 2 generated:
{
"name": "generate_image",
"body": {
"prompt": "A beautiful rainy night in a small village, with a raincoat wearing person holding an umbrella. In the style of Leonid Afremov."
},
"path_params": {},
"query_params": {}
}
Constant parts of api call (POST, GET, token, etc..) are excluded on purpose, as they are piped trough code.
So model is instructed to generate only "moving parts".
So what was better for me using it in this way, is that it much more often chooses right API Call, and avoiding it when is not needed for user query.
I am writing this as someone might find it useful...
First of all, Nous hermes 2 pro is a beast of an 7B reasoning model.
I've been experimenting with the Hermes 2 Pro, and I've noticed some maybe interesting behavior.
According to the repository instructions, I initially used the model with \
However, I discovered that it performs significantly better (at least for my use cases) when directly interacting with raw API calls (no tools defined) and no langchain.
Specifically, I set up each API endpoint in an OpenAPI specification and provided it in user part of the prompt, allowing the model to dynamically choose any endpoint it deems useful, while disregarding those it doesn't need.
It was able to use tools when needed, and avoid tools altogether when not needed even in long chat sessions.
Example "Tools" used: generate_image, search_vector_store, expand_image, etc...
Sample that Hermes 2 generated:
Constant parts of api call (POST, GET, token, etc..) are excluded on purpose, as they are piped trough code. So model is instructed to generate only "moving parts".
So what was better for me using it in this way, is that it much more often chooses right API Call, and avoiding it when is not needed for user query.
I am writing this as someone might find it useful...