France-Travail / gabarit

Gabarit : kickstart your data science project from scratch
GNU Affero General Public License v3.0
57 stars 11 forks source link

[NLP] Rework model_rules.py #33

Open LexABzH opened 2 years ago

LexABzH commented 2 years ago

Problem

Model Rules was created to perform 'predictions' with simple human rules. However this model did not get any update for quite a moment and needs to be updated to the same level of quality as the other models. Plus, it is not tested.

Concerned template

Solution

LexABzH commented 2 years ago

Model rules should be functions, but we can't find a reliable way to save them. If they are model's methods, we can pickle an instance but it won't protect it from changes in our code.

For example :

This is not acceptable has we want the model to have the same results whenever we reload it.

We thought about using sklearn's FunctionTransformer, but it has the same behavior (see https://github.com/scikit-learn/scikit-learn/issues/12903 and https://github.com/scikit-learn/scikit-learn/issues/12904)

LexABzH commented 1 year ago

Note : it works fine with lambda functions