JohnSnowLabs / langtest

Deliver safe & effective language models
http://langtest.org/
Apache License 2.0
488 stars 36 forks source link

Update dependencies in setup.py and pyproject.toml #981

Closed chakravarthik27 closed 6 months ago

chakravarthik27 commented 6 months ago

Description:

This PR introduces the seqeval package into our project. seqeval is a Python framework for sequence labeling evaluation. It can be used for any kind of sequential data and is particularly useful for tasks like Named Entity Recognition (NER), part-of-speech tagging, and word segmentation.

MicrosoftTeams-image (4)

Changes include:

  • Added seqeval to the pyproject.toml file.
  • Updated the poetry.lock file to reflect the new dependency.
  • The seqeval>=1.2.0 added is compatible with our current project setup and does not conflict with our existing dependencies.
  • seqeval package is added to evaluate as an extras package section in pyproject.toml image

Why is this change necessary?

seqeval will allow us to evaluate our sequence labeling models more effectively and it requires the HF callback feature in this library. It provides several useful metrics out of the box, such as precision, recall, and F1 score.

How has this been tested?

After adding seqeval, the project was built and all unit tests were run to ensure no existing functionality was broken and using HF_Callback_NER.ipynb

Potential Risks: As with any new dependency, there is a risk of conflicts with existing packages. However, the added package has been tested thoroughly to minimize this risk.