search

daveshap / ACE_Framework

ACE (Autonomous Cognitive Entities) - 100% local and open source autonomous agents
MIT License
1.45k stars 206 forks source link

PITCH: Testing framework #43

Closed thehunmonkgroup closed 9 months ago

thehunmonkgroup commented 1 year ago

PITCH: Testing framework

Problem

We need a testing framework.

Appetite

3 seconds

Solution

pytest

Rabbit Holes

Message delivery by carrier pigeon.

No-gos

pip uninstall pytest
import unittest
# Commit sins...
alanh90 commented 1 year ago

We should start off with our own version of the Turing test though. I have some of my own questions that ChatGPT fails at which could prove to the public that ours is more human in nature.

I have some ideas as well for further testing that we should discuss at some point past MVP stages. Im thinking a more cognitive approach for testing outputs. Having an AI analyze the output of ours to detect anything unusual. This would allow quick testing iterations and output suggestions. A more automated advanced turing test that also utilizes its own internal cognitive architecture, but built mainly for unnatrual/rude/etc speech detection. GPTs can pick up oddities to some degree if they are tasked with that purpose and if we combine that with other memory modules this could be interesting.

thehunmonkgroup commented 9 months ago

We're not using the PITCH structure anymore, closing.