CarperAI / cheese

Used for adaptive human in the loop evaluation of language and embedding models.
MIT License
303 stars 24 forks source link

Architext Alpha #27

Closed ehavener closed 2 years ago

ehavener commented 2 years ago

Details

This PR introduces preliminary support for Architext. The user interface is launched by running python -m examples.architext.architext_test. Temporary credentials output by this command are used to sign in. You may need to obtain a copy of the fine tuned gptj-162M model and load it locally before running.

The collected preferences are: prompt, creativity, result, feedback, and score. The result with selected rooms removed is also collected as a diff of the result dict and room selections.

Screenshots

Prompt input 127 0 0 1_7861_(iPad Mini)

Model output 127 0 0 1_7861_(iPad Mini) (1)

Preference collection 127 0 0 1_7861_(iPad Mini) (2)