JohnSnowLabs / langtest

Deliver safe & effective language models

http://langtest.org/

Apache License 2.0

490 stars 37 forks source link

Fix llm eval for transformers and lm studio and Code Refactoring #963

Closed Prikshit7766 closed 7 months ago

Prikshit7766 commented 7 months ago

Description

This PR includes the following enhancements and fixes:

Fixed llm_eval metric for Transformers and LM Studio models.
Refactored Sycophancy Sample
Added User Prompt for Sensitivity Category: Increased customization options for users.
Improved Sycophancy Transform Method: Integrated random name generation for professors.

➤ Fixes # (issue)

Type of change

Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Usage

Checklist:

[ ] I've added Google style docstrings to my code.
[ ] I've used pydantic for typing when/where necessary.
[ ] I have linted my code
[ ] I have added tests to cover my changes.

Screenshots (if appropriate):