JohnSnowLabs / langtest

Deliver safe & effective language models
http://langtest.org/
Apache License 2.0
501 stars 40 forks source link

Explore BYOD library #560

Open dcecchini opened 1 year ago

dcecchini commented 1 year ago

Explode the BYOD repository for additional tests or datasets to add to nlptest.

Examples:

JulesBelveze commented 1 year ago

My personal takeaways:

Even though there's nothing ground breaking in the repo and paper I do think it is really interesting to have an approach in which the model is evaluated against itself.

dcecchini commented 1 year ago

I agree, some of the tests are very simple, but also easy to implement and fast to run. So maybe we could add like the toxicity one for a quick test without any dependency to an external library to run an ML model...

Let's make a list of what is worth to bring to nlptest and add them to the roadmap.

dcecchini commented 1 year ago

I just found a paper about self evaluating, would interesting to read and check if we can implement it.

https://arxiv.org/abs/2306.13651?utm_source=substack&utm_medium=email