search

HumanSignal / label-studio-ml-backend

Configs and boilerplates for Label Studio's Machine Learning backend
Apache License 2.0
575 stars 260 forks source link

NER ml backend doesn't display predictions #176

Open Oussamayousre opened 2 years ago

Oussamayousre commented 2 years ago 
ℹ Saving to output directory:
/home/oussama/my-ml-backend0/checkpoints/20221114204115
ℹ Using CPU

=========================== Initializing pipeline ===========================
✔ Initialized pipeline

============================= Training pipeline =============================
ℹ Pipeline: ['tok2vec', 'ner']
ℹ Initial learn rate: 0.001
E    #       LOSS TOK2VEC  LOSS NER  ENTS_F  ENTS_P  ENTS_R  SCORE 
---  ------  ------------  --------  ------  ------  ------  ------
  0       0          0.00     11.94    0.00    0.00    0.00    0.00
200     200          0.68    206.59    0.00    0.00    0.00    0.00
400     400          0.00      0.00    0.00    0.00    0.00    0.00
✔ Saved pipeline to output directory
/home/oussama/my-ml-backend0/checkpoints/20221114204115/model-last
oussama2
[2022-11-14 20:41:24,480] [ERROR] [label_studio_ml.exceptions::exception_f::53] Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/exceptions.py", line 39, in exception_f
    return f(*args, **kwargs)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/api.py", line 93, in _train
    job = _manager.train(annotations, project, label_config, **params)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 714, in train
    cls.get_or_create(project, label_config, force_reload=True, train_output=train_output)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 490, in get_or_create
    if not cls.has_active_model(project) or \
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 452, in has_active_model
    return cls._key(project) in cls._current_model
TypeError: argument of type 'ModelWrapper' is not iterable

Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/exceptions.py", line 39, in exception_f
    return f(*args, **kwargs)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/api.py", line 93, in _train
    job = _manager.train(annotations, project, label_config, **params)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 714, in train
    cls.get_or_create(project, label_config, force_reload=True, train_output=train_output)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 490, in get_or_create
    if not cls.has_active_model(project) or \
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 452, in has_active_model
    return cls._key(project) in cls._current_model
TypeError: argument of type 'ModelWrapper' is not iterable

[2022-11-14 20:41:24,480] [INFO] [werkzeug::_log::225] 127.0.0.1 - - [14/Nov/2022 20:41:24] "POST /train HTTP/1.1" 500 -
[2022-11-14 20:41:24,504] [INFO] [werkzeug::_log::225] 127.0.0.1 - - [14/Nov/2022 20:41:24] "GET /health HTTP/1.1" 200 -
[2022-11-14 20:41:24,509] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668451470 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:24,510] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668451450 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:24,510] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668451120 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:24,511] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668451116 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:24,511] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668451088 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:24,511] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668451068 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:24,511] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668450840 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:24,511] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668450835 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:24,511] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668450218 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:24,512] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668450217 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:24,512] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668450211 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:24,512] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668450210 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:24,512] [INFO] [model::__init__::55] MODEL CHECKPOINT: fallback
[2022-11-14 20:41:24,513] [INFO] [werkzeug::_log::225] 127.0.0.1 - - [14/Nov/2022 20:41:24] "POST /setup HTTP/1.1" 200 -
[2022-11-14 20:41:31,135] [INFO] [werkzeug::_log::225] 127.0.0.1 - - [14/Nov/2022 20:41:31] "GET /health HTTP/1.1" 200 -
[2022-11-14 20:41:31,140] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668451470 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:31,141] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668451450 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:31,141] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668451120 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:31,141] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668451116 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:31,141] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668451088 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:31,141] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668451068 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:31,141] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668450840 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:31,141] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668450835 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:31,141] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668450218 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:31,142] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668450217 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:31,142] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668450211 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:31,142] [ERROR] [label_studio_ml.model::get_result_from_last_job::131] 1668450210 job returns exception: 
Traceback (most recent call last):
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 129, in get_result_from_last_job
    result = self.get_result_from_job_id(job_id)
  File "/home/oussama/Downloads/label-studio-ml-backend/label_studio_ml/model.py", line 111, in get_result_from_job_id
    assert isinstance(result, dict)
AssertionError
[2022-11-14 20:41:31,142] [INFO] [model::__init__::55] MODEL CHECKPOINT: fallback
[2022-11-14 20:41:31,143] [INFO] [werkzeug::_log::225] 127.0.0.1 - - [14/Nov/2022 20:41:31] "POST /setup HTTP/1.1" 200 -`

these are the data I'm using, only to test if everything is working

[ { "data": { "text": "More rapid margin progress." }} , { "data": { "text": "Demonstration of gross margin and EBITA margin progression. Better than GDP+ growth." }} ]

It seems that once the model is trained, the backend isn't able to capture the predicted data, Please if anyone can help I will highly appreciate it

KonstantinKorotaev commented 1 year ago

Hi @Oussamayousre What version of ML backend are you using?