ContextLab / chatify

Add ipython magic commands to Jupyter notebooks that provide LLM-driven enhancements

MIT License

22 stars 10 forks source link

Tests [TODO] #9

Open jeremymanning opened 1 year ago

jeremymanning commented 1 year ago

Overview

Write unit tests for each function

Completion criteria

[ ] For each function, add a "todo" issue to create a test for the function, describing (a) what the function is supposed to do and (b) how it should be tested. If there are any edge cases or tricky scenarios to watch out for, those should be noted too

Resources (optional)

pytest: framework for writing and running tests
GitHub actions: can be used to automate tests when pull requests are merged
davos: archive contains useful code for running tests that involve notebooks

Discussion points

Adding issues doesn't require coding, but it'd be good to have at least one "coding" person work on this task to help define useful tests.

HemuManju commented 1 year ago

Langchain testing https://python.langchain.com/en/latest/modules/models/llms/examples/fake_llm.html?highlight=fake

xuyi0619 commented 1 year ago

Thank you so much !

HemuManju commented 1 year ago

@xuyi0619, hope you are doing well! I think the project is reaching a critical mass where we need extensive testing of the modules. Can we collaborate on this and write test cases?

xuyi0619 commented 1 year ago

Hi @HemuManju , of course！Let’s meet on zoom. I have time tomorrow, and next Tuesday and Thursday. Let me know which date and time works for you.

HemuManju commented 1 year ago

I am available on Tuesday.

xuyi0619 commented 1 year ago

Ok, I will send you the zoom link when I get back.

xuyi0619 commented 1 year ago

Grace Xu is inviting you to a scheduled Zoom meeting.

Topic: Grace Xu's Personal Meeting Room

Join Zoom Meeting https://us04web.zoom.us/j/3252265589?pwd=eG1DVjRHSWtFQXFnOE5hK2NqRlgrZz09

Meeting ID: 325 226 5589 Passcode: TuXSp9

I am available on Tuesday.

HemuManju commented 1 year ago

Thanks @xuyi0619, at time are we meeting? I am on west coast.

xuyi0619 commented 1 year ago

How about 1pm?

On Sun, May 21, 2023 at 8:06 PM Hemu @.***> wrote:

Thanks @xuyi0619 https://github.com/xuyi0619, at time are we meeting? I am on west coast.

— Reply to this email directly, view it on GitHub https://github.com/ContextLab/chatify/issues/9#issuecomment-1556336910, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3QX6NTEFW3PLFLN2276UJTXHKUXHANCNFSM6AAAAAAXN6ONYI . You are receiving this because you were mentioned.Message ID: @.***>

HemuManju commented 1 year ago

Works for me! Looking forward for the meeting.

xuyi0619 commented 1 year ago

@HemuManju I am on zoom.

jeremymanning commented 1 year ago

Some tests are hard to write, and others seem easier.

Hard tests

Making sure the interface renders correctly
Making sure the prompts are being sent to the LLM correctly
Testing the efficacy of the prompts
Checking formatting stuff (e.g., syntax highlighting, markdown rendering)

Easier tests

Verify that each type of LLM can be created without crashing, and can produce string outputs when called
Verify that a Chatify object can be created without crashing