search

Libr-AI / OpenFactVerification

Loki: Open-source solution designed to automate the process of verifying factuality
https://loki.librai.tech/
MIT License
1.03k stars 45 forks source link

Get token count for FactCheck().check_response #11

Closed evangriffiths closed 7 months ago

evangriffiths commented 7 months ago

Usage:

import sys
sys.path.append("/Users/evan/dev/OpenFactVerification")
from dotenv import load_dotenv
from factcheck import FactCheck

load_dotenv(dotenv_path="/Users/evan/dev/OpenFactVerification/.env")
results = FactCheck().check_response("The sky is green")
print(results["token_count"])

Gives

{'num_raw_tokens': 4, 'num_checkworthy_tokens': 5, 'total_prompt_tokens': 1748, 'total_completion_tokens': 231}
evangriffiths commented 7 months ago

Hacky PR to check token counts. Not intending to merge, so closing