LAION-AI / Open-Assistant

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.
https://open-assistant.io
Apache License 2.0
37.04k stars 3.23k forks source link

OA Long Context Developer Meeting #3552

Open andreaskoepf opened 1 year ago

andreaskoepf commented 1 year ago

When & Where

Long-context Techniques

We would like to hear your opinion:

Evaluation

In order to properly compare the different long-context techniques we should agree on some evaluation metrics to use.

Candidate metrics:

Ideally we would use the same evaluation code or at least if the evaluation code would be made available open-source to repeat the experiments.