Open faizanahemad opened 5 years ago
Break this into Subtasks if this seems too big to do by 1 person or in 1 week.
Facing a weird issue with scikit-learn (0.21.3)
While importing precision_recall_fscore_support
from inside a file in fastnet/inspect
getting an error, but it seems to work while calling the same from Python shell.
Can anyone try replicating this once?
from sklearn.metrics import precision_recall_fscore_support
Stacktrace which I'm getting -
Traceback (most recent call last):
File "/Users/dhruv-seam/Documents/FastNet/fastnet/inspect/gradcam.py", line 1, in <module>
from sklearn.metrics import precision_recall_fscore_support, confusion_matrix, balanced_accuracy_score, accuracy_score
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/sklearn/__init__.py", line 76, in <module>
from .base import clone
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/sklearn/base.py", line 16, in <module>
from .utils import _IS_32BIT
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/sklearn/utils/__init__.py", line 17, in <module>
from . import _joblib
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/sklearn/utils/_joblib.py", line 8, in <module>
import joblib
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/joblib-0.14.0-py3.7.egg/joblib/__init__.py", line 113, in <module>
from .memory import Memory, MemorizedResult, register_store_backend
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/joblib-0.14.0-py3.7.egg/joblib/memory.py", line 32, in <module>
from ._store_backends import StoreBackendBase, FileSystemStoreBackend
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/joblib-0.14.0-py3.7.egg/joblib/_store_backends.py", line 16, in <module>
from .backports import concurrency_safe_rename
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/joblib-0.14.0-py3.7.egg/joblib/backports.py", line 9, in <module>
from pkg_resources import parse_version
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3126, in <module>
@_call_aside
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3110, in _call_aside
f(*args, **kwargs)
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3139, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 572, in _build_master
ws = cls()
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 565, in __init__
self.add_entry(entry)
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 621, in add_entry
for dist in find_distributions(entry, True):
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1873, in find_distributions
finder = _find_adapter(_distribution_finders, importer)
File "/Users/dhruv-seam/Documents/FastNet/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3039, in _find_adapter
types = _always_object(inspect.getmro(getattr(ob, '__class__', type(ob))))
AttributeError: module 'inspect' has no attribute 'getmro'
Proposed Features
Class Distribution see here
Confusion Matrix see here
Table of Per Class Precision-Recall-Total Counts See
inspect_predictions
Plot Acc and loss curves from Model See
plot_model_history
Show Ordered Misclassifiec classes. The classes which are most mis-classified will come on top and help us focus on those. Logic of identifying which class is most mis-classified should be based.
Show Ordered Correctly classified classes. Logic similar to above.
Show Ordered Actual Class Misclassified As
Another Class
Ordered. Idea: Take actual class as dog, Now dog is misclassified as cat 5%, as ship 2%, as frog 1%. Make a table of this. Input is name of actual class. Output is a table with cat-5% at the top.Any Other way of interpreting/visualising Confusion Matrix
Any other way of interpreting Errors/Predictions
Put this under https://github.com/faizanahemad/FastNet/tree/master/fastnet/inspect , make another file called
model_inspection.py
so you don't overlap with someone else's workHow the API looks like?
Intended Usage Once model is trained this function can be called with Model and dataset to visualize.
Limitations
Online References/Documentation was referred while making this API
Branch Name (The github branch in which you are working on this).
Note: Before raising a PR make sure that you have made an issue like this with tag: enhancement, ensure your code is tested and documented.