search

dimitrismistriotis / alt-profanity-check

A fast, robust library to check for offensive language in strings, dropdown replacement of "profanity-check".
https://pypi.org/project/alt-profanity-check/
MIT License
68 stars 15 forks source link

Version mismatch in pickled data #15

Closed bgoldowsky closed 1 year ago

bgoldowsky commented 2 years ago

Using the 1.0.2.1 version, a set of warnings are issued when this library is imported. It appears that the trained data was created with a different version of scikit-learn than the one that it pinned in requirements.txt.

>>> from profanity_check import predict, predict_prob
/Users/borisgoldowsky/idea/clusive/venv/lib/python3.9/site-packages/sklearn/base.py:329: UserWarning: Trying to unpickle estimator TfidfTransformer from version 1.0.1 when using version 1.0.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/modules/model_persistence.html#security-maintainability-limitations
  warnings.warn(
/Users/borisgoldowsky/idea/clusive/venv/lib/python3.9/site-packages/sklearn/base.py:329: UserWarning: Trying to unpickle estimator TfidfVectorizer from version 1.0.1 when using version 1.0.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/modules/model_persistence.html#security-maintainability-limitations
  warnings.warn(
/Users/borisgoldowsky/idea/clusive/venv/lib/python3.9/site-packages/sklearn/base.py:329: UserWarning: Trying to unpickle estimator LinearSVC from version 1.0.1 when using version 1.0.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/modules/model_persistence.html#security-maintainability-limitations
  warnings.warn(
/Users/borisgoldowsky/idea/clusive/venv/lib/python3.9/site-packages/sklearn/base.py:329: UserWarning: Trying to unpickle estimator _SigmoidCalibration from version 1.0.1 when using version 1.0.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/modules/model_persistence.html#security-maintainability-limitations
  warnings.warn(
/Users/borisgoldowsky/idea/clusive/venv/lib/python3.9/site-packages/sklearn/base.py:329: UserWarning: Trying to unpickle estimator CalibratedClassifierCV from version 1.0.1 when using version 1.0.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/modules/model_persistence.html#security-maintainability-limitations
  warnings.warn(
>>>
dimitrismistriotis commented 1 year ago

Hi @bgoldowsky think this happened in a version and we created a follow up version immediately.

We just released last version today where things should be OK, is there a reason you cannot upgrade?

As a follow up I will "yank" this version on PyPi which will make it not recommended unless pinned.

bgoldowsky commented 1 year ago

@dimitrismistriotis thank you so much for this answer - which took me a long time to see since it ended up in my spam email box :(

I'm unable to update to the 1.1.x versions since they drop compatibility with Python 3.7, which out application wants to support.

With 1.0.2.1 yanked, so I need to downgrade to 1.0.2 ?

dimitrismistriotis commented 1 year ago

You can still install every version as I understood, yanked means not supported. 1.0.2.1 had some fixes from errors we discovered on 1.0.2. We were somehow careless on that one.

I do not know why you cannot upgrade Python versions, I recently discovered the deprecation schedule. 3.7 will be still supported for about 6 months from when I am typing these lines, see https://peps.python.org/pep-0537/ and https://www.python.org/downloads/. It was a surprise to me as I thought they were kept maintained longer.