patham9 / NarsGPT

A NARS implemented as a GPT model prompted to invoke reasoning steps, with NARS-based memory and control machinery implemented in Python.
MIT License
40 stars 15 forks source link

Evaluation suite #4

Open patham9 opened 1 year ago

patham9 commented 1 year ago

An initial evaluation suite for incremental open-ended Q&A is available in https://github.com/patham9/NarsGPT/tree/gptONA mem.json symbolic-vector hybrid database file from running the incremental Q&A can be obtained here: https://github.com/patham9/NarsGPT/releases/tag/Evaluation_Memfile This allows interactive Q&A testing with the belief learned from the csv.

Special thanks to Peter Voss for providing an incremental Q&A benchmark: https://github.com/patham9/NarsGPT/blob/Evaluation/INT_Inf_benchmarkTest.csv