Add an LLMPipelineTest

[kuraisle]

Is this the right issue type?

• [x] Yes, I'm planning work for this project team.

Summary

We're testing different LLMs for their performance on the drug dataset. For the evaluation framework, we need an LLMPipelineTest class that inherits from SingleResultPipelineTest.

It needs to be compatible with LLMPipeline (#60) and ExactMatchMetric.

Calling .run_pipeline() on some input data should return a list of single strings.

Calling .evaluate() should compare the output of an LLMPipeline with the desired input.

Could we have a commit for each task to make it clearer what's happening, please?

Acceptance Criteria

• [ ] Carrot-Assistant/evaluation/pipeline_tests.py present
• [ ] LLMPipelineTest class created in pipeline_tests.py
• [ ] LLMPipelineTest .run_pipeline() passes tests
• [ ] LLMPipelineTest,.evaluate() passes tests

Tasks

• [ ] create file for pipeline tests
• [ ] create LLMPipelineTest class
  • [ ] initialises with an LLMPipeline
  • [ ] initialises with [ExactMatchMetric]
• [ ] implement LLMPipelineTest.run_pipeline(*args)
• [ ] implement LLMPipelineTest.evaluate(*args)
• [ ] LLMPipelineTest tests
  • [ ] .run_pipeline() returns a list of strings with the same length as input
  • [ ] .evaluate() returns correctly
• [ ] Documentation.

Confirm creation

• [x] This issue is ready