SeldonIO / alibi

Algorithms for explaining machine learning models
https://docs.seldon.io/projects/alibi/en/stable/
Other
2.4k stars 252 forks source link

How to pass parameters to the /api/v1.0/explain using AnchorImage? #946

Open abhaygargab opened 1 year ago

abhaygargab commented 1 year ago

Hi Team,

I am able to use and pass parameters to AnchorImage explain method like below in my jupyter notebook:

explanation = explainer.explain(image, threshold=.7, p_sample=.5, 
                                tau=0.25, batch_size = 10, 
                                beam_size = 1, stop_on_first = True,
                                verbose = True, verbose_every = 2,
                                n_covered_ex = 1)

However, after deploying the explainer following the below link, I am not able to get pass the parameters to the explain method through the /api/v1.0/explain api: https://docs.seldon.io/projects/seldon-core/en/latest/examples/iris_explainer_poetry.html

def explain_fn(X, 
               model_url = ("http://10.113.34.101/seldon/seldon/deploymentname-explainer/model/api/v1.0/explain")):
    data = {"data": {"ndarray": X.tolist()}, "meta": {"params": {"threshold": 0.7}}}
    r = requests.post(model_url, json=data)
    return r.json()

dummy_inp = np.ones((1, 224, 224, 3))
explanation = explain_fn(dummy_inp)

I have tried various ways like:

1.

   data = {"data": {"names": ["image", "threshold"], "ndarray": X.tolist(), "float_val": 0.5}}
  1. data = {"data": {"ndarray": X.tolist(), "float_val": 0.5,
                      "float_val": 0.5, "float_val": 0.1,
                      "float_val": 0.15, "int_val": 10,
                      "int_val": 10000, "int_val": 1,
                      "bool_val": True}}

but only the default parameters to the explain function are used always.

explanation["meta"]["params"]
{'custom_segmentation': False,
 'segmentation_kwargs': {'n_segments': 10,
  'compactness': 10,
  'sigma': 0.5,
  'start_label': 0},
 'p_sample': 0.5,
 'seed': None,
 'image_shape': [224, 224, 3],
 'images_background': None,
 'segmentation_fn': 'slic',
 'threshold': 0.95,
 'tau': 0.15,
 'batch_size': 100,
 'beam_size': 1,
 'stop_on_first': False,
 'verbose': False,
 'verbose_every': 1,
 'n_covered_ex': 10,
 'delta': 0.1,
 'coverage_samples': 10000,
 'max_anchor_size': None,
 'min_samples_start': 100,
 'binary_cache_size': 10000,
 'cache_margin': 1000,
 'kwargs': {}}

Can you please guide me to ways to pass parameters while calling the explainers through /api/v1.0/explain ?

ascillitoe commented 1 year ago

*editing for clarity

ascillitoe commented 1 year ago

Hi @abhaygargab, would you be able to reopen this issue in the seldon-core repo please?

I believe you may be better off using the newer V2 protocol (see example). You should be able to set static explain kwarg's in your explainer CRD yaml file then. Someone with the seldon-core team will be able to confirm the proper format for this with you.