search

CarperAI / cheese

Used for adaptive human in the loop evaluation of language and embedding models.
MIT License
303 stars 24 forks source link

Issues with Running Examples #53

Open mshuffett opened 1 year ago

mshuffett commented 1 year ago

Describe the bug

There are several issues with trying to run the examples from scratch (on an M1 Mac). I would submit a PR to solve these but I probably don't have time at the moment.

To Reproduce

Try to follow the existing README on an M1 Mac.

Issues and Solutions

RabbitMQ won't run correctly using Conda

Solution

Add comment to README mentioning you can also run this command to run RabbitMQ docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3.11-management Taken from https://www.rabbitmq.com/download.html

It is unexpected for the user to get a password required when running the example and there is no explanation of how to proceed

Solution

Initially add a comment to README describing that the username and password are printed on the command line. Long term, likely don't use public by default for the examples OR provide a way to disable that using a command line argument or environment variable.

The instruct_hf_pipeline.py doesn't work out of the box because the dependencies are not in the requirements.txt file

Solution

Either add the dependencies to requirements.txt or add it in the README or print a more helpful message when running that example.