JohnSnowLabs / langtest

Deliver safe & effective language models

http://langtest.org/

Apache License 2.0

502 stars 40 forks source link

Refactor/add configurable parameters #866

Closed alytarik closed 1 year ago

alytarik commented 1 year ago

Description

This PR adds configurable parameters instead of some hardcoded values in

CrowS-Pairs
- diff_treshold
- filter_treshold
Wino-Bias
- diff_treshold
Clinical, Disinformation, Factuality,Translation
- sentence_transformer

➤ Fixes #857

Type of change

[ ] New feature (non-breaking change which adds functionality)

Usage

tests:
  wino-bias:
    gender-occupational-stereotype:
      min_pass_rate: 0.70
      diff_threshold: 0.03

tests:
  crows-pairs:
    common-stereotypes:
      min_pass_rate: 0.70
      diff_threshold: 0.10
      filter_threshold: 0.15

Checklist:

[x] I've added Google style docstrings to my code.
[x] I've used pydantic for typing when/where necessary.
[x] I have linted my code
[x] I have added tests to cover my changes.

Screenshots (if appropriate):