oegedijk / explainerdashboard

Quickly build Explainable AI dashboards that show the inner workings of so-called "blackbox" machine learning models.
http://explainerdashboard.readthedocs.io
MIT License
2.3k stars 331 forks source link

Updating to 0.2.19 broke custom dashboard #61

Closed hkoppen closed 3 years ago

hkoppen commented 3 years ago

After updating to the most recent version, my custom dashboard isn't working anymore, neither after "inplace construction" nor after loading from disk. More precisely:

test_explainer = RegressionExplainer(dec_tree, X_small, y_small, cats=['Day_of_week', 'Hour', 'Vehicle', 'Position'])
ExplainerDashboard(test_explainer).run()

works fine but

explainer = RegressionExplainer(dec_tree, X_small, y_small, cats=['Day_of_week', 'Hour', 'Vehicle', 'Position'])
db = ExplainerDashboard(explainer, [VecPos, CustomFeatImpTabv2, CustomWhatIfTabv2])

yields

Building ExplainerDashboard..
Detected notebook environment, consider setting mode='external', mode='inline' or mode='jupyterlab' to keep the notebook interactive while the dashboard is running...
Generating layout...
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-d2ec058da33d> in <module>
      1 db = ExplainerDashboard(explainer,
----> 2                         [VecPos, CustomFeatImpTabv2, CustomWhatIfTabv2],
      3                         # external_stylesheets=[FLATLY],
      4                         # title='Predictive Maintenance'
      5                        )

c:\users\hkoppen\...\site-packages\explainerdashboard\dashboards.py in __init__(self, explainer, tabs, title, name, description, hide_header, header_hide_title, header_hide_selector, hide_poweredby, block_selector_callbacks, pos_label, fluid, mode, width, height, bootstrap, external_stylesheets, server, url_base_pathname, responsive, logins, port, importances, model_summary, contributions, whatif, shap_dependence, shap_interaction, decision_trees, **kwargs)
    510                             block_selector_callbacks=self.block_selector_callbacks,
    511                             pos_label=self.pos_label,
--> 512                             fluid=fluid))
    513         else:
    514             tabs = self._convert_str_tabs(tabs)

c:\users\hkoppen\...\site-packages\explainerdashboard\dashboards.py in __init__(self, explainer, tabs, title, description, header_hide_title, header_hide_selector, hide_poweredby, block_selector_callbacks, pos_label, fluid, **kwargs)
    128 
    129         self.selector = PosLabelSelector(explainer, name="0", pos_label=pos_label)
--> 130         self.tabs  = [instantiate_component(tab, explainer, name=str(i+1), **kwargs) for i, tab in enumerate(tabs)]
    131         assert len(self.tabs) > 0, 'When passing a list to tabs, need to pass at least one valid tab!'
    132 

c:\users\hkoppen\...\site-packages\explainerdashboard\dashboards.py in <listcomp>(.0)
    128 
    129         self.selector = PosLabelSelector(explainer, name="0", pos_label=pos_label)
--> 130         self.tabs  = [instantiate_component(tab, explainer, name=str(i+1), **kwargs) for i, tab in enumerate(tabs)]
    131         assert len(self.tabs) > 0, 'When passing a list to tabs, need to pass at least one valid tab!'
    132 

c:\users\hkoppen\...\site-packages\explainerdashboard\dashboards.py in instantiate_component(component, explainer, name, **kwargs)
     64 
     65     if inspect.isclass(component) and issubclass(component, ExplainerComponent):
---> 66         component = component(explainer, name=name, **kwargs)
     67         return component
     68     elif isinstance(component, ExplainerComponent):

TypeError: __init__() got an unexpected keyword argument 'name'

Offtopic: I am going to deploy the dashboard using Docker in one or two weeks, hopefully I am able to give some feedback on the respective issue then.

hkoppen commented 3 years ago

A small example using the example in the docs any my regression data:

[...]  # Some packages

from explainerdashboard import RegressionExplainer, ExplainerDashboard
from explainerdashboard.custom import *

[...]  # Load data & model

explainer = RegressionExplainer(dec_tree, X_small, y_small, cats=['Day_of_week', 'Hour', 'Vehicle', 'Position'])

class CustomDashboard(ExplainerComponent):
    def __init__(self, explainer, **kwargs):
        super().__init__(explainer, title="Custom Dashboard")
        self.contrib = ShapContributionsGraphComponent(explainer,
                            hide_selector=True, hide_cats=True, hide_depth=True, hide_sort=True,)
        self.register_components()

        def layout(self):
            return dbc.Container([
                dbc.Row([
                    dbc.Col([
                        self.contrib.layout(),
                    ])
                ])
            ])

db = ExplainerDashboard(explainer, CustomDashboard)

returns basically the same error as above.

oegedijk commented 3 years ago

Ah, yes, I can see what went wrong here. You can fix it by adding name=None to the __init__ of your custom tab.

Basically in order to get rid of the random uuid names which cause trouble when you deploy with a docker swarm, I give every tab an incremental name, so the first tab gets name='1', the second name='2', etc. But this only works ofcourse when the init actually takes the name parameter. Which is true for the default composites, but not always for custom ones.

So I can either:

  1. Give clear documentation instruction to always include name as a parameter in the init
  2. Somehow detect the signature of the custom dashboard and see if it takes name as a a parameter..
oegedijk commented 3 years ago

although in the second example you actually accept **kwargs, so then it should have worked, no?

oegedijk commented 3 years ago

It actually seems that somehow _store_child_params is no longer storing parameters to attributes, which is why self.name cannot be found. Am investigating!

oegedijk commented 3 years ago

Just pushed v0.2.19.1 which should fix your issue. Also is now respectful of ExplainerComponents that do not take **kwargs:

if inspect.isclass(component) and issubclass(component, ExplainerComponent):
        init_argspec = inspect.getargspec(component.__init__)
        if not init_argspec.keywords:
            kwargs = {k:v for k,v in kwargs.items() if k in init_argspec.args}
        if "name" in init_argspec.args:
            component = component(explainer, name=name, **kwargs)
        else:
            print(f"ExplainerComponent {component} does not accept a name parameter, "
                    f"so cannot assign name={name}!"
                    "Make sure to set name explicitly yourself if you want to "
                    "deploy across multiple workers or a cluster, as otherwise "
                    "each instance in the cluster will generate its own random "
                    "uuid name!")
            component = component(explainer, **kwargs)
        return component
hkoppen commented 3 years ago

It did!