phi-grib / flame

Modeling framework for eTRANSAFE project
GNU General Public License v3.0
12 stars 10 forks source link

GUI fails to detect model list after MCC returns nan #206

Closed EMVGaron closed 3 years ago

EMVGaron commented 3 years ago

After creating a model with lots of entry points (> 90000) and a high imbalance (ratio 0.004 pos/neg), MCC is returned as nan and thus, the GUI fails to recover the model list.

Steps to reproduce

  1. Create the model using Flame CLI: flame -a new -c manage -e aid_845 flame -f aid_845.sdf -e aid_845 -c build
  2. The model numbers appear in the terminal:
    INFO - Model finished successfully
    INFO -        nobj ( number of objects ) : 91641.0
    INFO -        nvarx ( number of predictor variables ) : 232.0
    INFO -        model ( model type ) : RF
    INFO -        TP_f ( True positives in fitting ) : 387.0
    INFO -        TN_f ( True negatives in fitting ) : 91254.0
    INFO -        FP_f ( False positives in fitting ) : 0.0
    INFO -        FN_f ( False negatives in fitting ) : 0.0
    INFO -        Sensitivity_f ( Sensitivity in fitting ) : 1.0
    INFO -        Specificity_f ( Specificity in fitting ) : 1.0
    INFO -        MCC_f ( Matthews Correlation Coefficient in fitting ) : 1.0
    INFO -        TP ( True positives in cross-validation ) : 0.0
    INFO -        TN ( True negatives in cross-validation ) : 91254.0
    INFO -        FP ( False positives in cross-validation ) : 0.0
    INFO -        FN ( False negatives in cross-validation ) : 387.0
    INFO -        Sensitivity ( Sensitivity in cross-validation ) : 0.0
    INFO -        Specificity ( Specificity in cross-validation ) : 1.0
    INFO -        MCC ( Matthews Correlation Coefficient in cross-validation ) : nan
  3. After creating the model, open Flame API: cd flame_API/flame_api python manage.py runserver
  4. Copy in the browser the host direction: 127.0.0.1:8000/
  5. The error appears in a pop-up: Screenshot from 2021-06-01 10-53-39
  6. In the terminal appears the following:
    Internal Server Error: /api/v1/manage/models
    Traceback (most recent call last):
    File "/home/eric/miniconda3/envs/flame/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
    File "/home/eric/miniconda3/envs/flame/lib/python3.6/site-packages/django/core/handlers/base.py", line 145, in _get_response
    response = self.process_exception_by_middleware(e, request)
    File "/home/eric/miniconda3/envs/flame/lib/python3.6/site-packages/django/core/handlers/base.py", line 143, in _get_response
    response = response.render()
    File "/home/eric/miniconda3/envs/flame/lib/python3.6/site-packages/django/template/response.py", line 106, in render
    self.content = self.rendered_content
    File "/home/eric/miniconda3/envs/flame/lib/python3.6/site-packages/rest_framework/response.py", line 70, in rendered_content
    ret = renderer.render(self.data, accepted_media_type, context)
    File "/home/eric/miniconda3/envs/flame/lib/python3.6/site-packages/rest_framework/renderers.py", line 103, in render
    allow_nan=not self.strict, separators=separators
    File "/home/eric/miniconda3/envs/flame/lib/python3.6/site-packages/rest_framework/utils/json.py", line 25, in dumps
    return json.dumps(*args, **kwargs)
    File "/home/eric/miniconda3/envs/flame/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
    File "/home/eric/miniconda3/envs/flame/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
    File "/home/eric/miniconda3/envs/flame/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
    ValueError: Out of range float values are not JSON compliant

Proposed solution Check in which point the calculation of the MCC returns a nan

EMVGaron commented 3 years ago

Problem has been solved. Now MCC returns 0.0 instead of nan and the GUI can load the model list without problems.

Closed issue