crossid / accessbot

Streamline resource access grants with AI-Powered chatbot
https://www.crossid.io
Other
2 stars 0 forks source link

Move Rule Engine LLM to a smaller LLM #148

Closed ErezSha closed 4 months ago

ErezSha commented 4 months ago

In order to reduce latency, try moving the rule engine LLM to a smaller & faster LLM (like gpt-4o)

  1. see how it works.
  2. check the latency is cut down
ErezSha commented 4 months ago

gpt-3.5 should be enough for the entry point node gpt-4o should be enough for rule engine node