Local LLM Deployment Optimization

amazon-science / RefChecker

RefChecker provides automatic checking pipeline and benchmark dataset for detecting fine-grained hallucinations generated by Large Language Models.

Apache License 2.0

281 stars 24 forks source link

Open void-b583x2-NULL opened 8 months ago

void-b583x2-NULL commented 8 months ago

Description of changes:

More arguments controlling the params for locally deployed LLM (Currently Mistral/Mixtral)
Corrected prompt for Mistral (to avoid truncations)
API optimization

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.