Test cases for a Chainlit app

[aharth]

I would like to write test cases for conversations with my Chainlit app.

The documentation on the website (https://docs.chainlit.io/advanced-features/test-debug) is a bit sparse and just mentions that "you will need to provide the Chainlit context to your test suite".

So I have added the following to my app.py.

if __name__ == "__main__":
    from chainlit.cli import run_chainlit
    run_chainlit(__file__)

I would like to be able to do something like the following in test.py.

import app

message = cl.Message(content="hello world")
print(f"Sending {message}")
app.chat(message)

Now when I run test.py, I get an exception related to the Chainlit context.

LookupError: <ContextVar name='chainlit' at 0x7f01a188f650>
chainlit.context.ChainlitContextException: Chainlit context not found

I have searched in the Chainlit source and in the cookbook for example code related to testing apps but could not find any.

Is it even possible to write such test cases with Chainlit?

[dosubot[bot]]

Hey there, @aharth! I'm here to help you with your Chainlit app testing issue. I'm a bot that can help you solve bugs, answer questions, and become a contributor. Let's tackle this together!

It seems like you're encountering an issue with the Chainlit context when trying to write test cases for your app. The ChainlitContextException indicates that the context is not being properly initialized before you attempt to use it in your test.

To address this, you can use the init_http_context or init_ws_context functions to initialize the context in your test environment. Additionally, you can use unittest.mock.Mock to create mock sessions and emitters for testing purposes. Here's a basic example of how you might set up a test case:

import unittest
from unittest.mock import Mock
import chainlit as cl
from chainlit.context import init_http_context, ChainlitContext

class TestChainlitApp(unittest.TestCase):
    def setUp(self):
        # Create a mock session and initialize the context
        mock_session = Mock()
        self.context = init_http_context(mock_session)

    def test_message_handling(self):
        # Create a test message
        message = cl.Message(content="hello world", author="User")

        # Simulate handling the message
        # You would replace this with the actual function you want to test
        # For example, app.chat(message) if that's your function
        response = self.handle_message(message)

        # Assert the expected outcome
        self.assertEqual(response.content, "Expected response")

    def handle_message(self, message):
        # This is a placeholder for your actual message handling logic
        # You can call your app's functions here
        return cl.Message(content="Expected response")

if __name__ == "__main__":
    unittest.main()

This example demonstrates how to set up a test case with a mock context. You can replace the handle_message function with your actual message handling logic. Make sure to initialize the context properly before running your tests to avoid the ChainlitContextException ^[1][2].

To continue talking to Dosu, mention @dosu.

[ilkersigirci]

I would like to know this, too. Testing is very crucial but it is not documented at all

[aharth]

That is how far I got trying to bypass the decorators to call the original function, but no cigar.

def dummy_decorator(func):
    return func

cl.step = dummy_decorator
cl.on_message = dummy_decorator
cl.on_chat_start = dummy_decorator