JohnSnowLabs / langtest

Deliver safe & effective language models

http://langtest.org/

Apache License 2.0

490 stars 38 forks source link

Fix: QA, Default Config, and Transformer Model for QA #926

Closed Prikshit7766 closed 9 months ago

Prikshit7766 commented 9 months ago

Description

Fix the QA Task
Standardize Config Parameters
Fix Transformer Model for QA
Removed langchain (PromptTemplate) dependency for Transformer Model
Sensitivity Test to New Language Models
Fix Evaluation in SycophancySample
Added Test for nlp task supported in langtest and Transformer Model for QA

Fixes

930

846

Type of change

Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Usage

Checklist:

[ ] I've added Google style docstrings to my code.
[ ] I've used pydantic for typing when/where necessary.
[ ] I have linted my code
[ ] I have added tests to cover my changes.