search

AI4LIFE-GROUP / OpenXAI

OpenXAI : Towards a Transparent Evaluation of Model Explanations
https://open-xai.github.io/
MIT License
227 stars 37 forks source link

Add api.py for external API calls to OpenXAI #1

Closed jiaqima closed 1 year ago

jiaqima commented 1 year ago

Description

Implemented an OpenXAI class in api.py to serve for external API calls.

Example Usage

from openxai.api import OpenXAI

oxai = OpenXAI(data_name="german", model_name="ann", explainer_name="lime")

# query full data
df_full = oxai.query()

# query a batch of data with feature tensor `X` and label tensor `y`
df_batch = oxai.query(X, y)

The returned df_full or df_batch are pandas dataframes with each row corresponds to a data sample. The number of columns is 2d + 3, where d is the feature dimension.

The columns from left to right are: features (d), feature attribution scores (d), label (1), predicted label (1), and is_test flag (1).

Test

The code has been successfully tested on 3 datasets ("compas", "adult", "german"), 2 models ("ann", "lr"), and 6 explainers ("grad", "sg", "itg", "ig", "shap", "lime").

To reproduce the test, run python openxai/api.py under the root of this repo.

Note: Currently there seems to be path issues for this API to be used externally. The issue is possibly due to the path usage by LoadModel, which should be fixed in a separate PR.

y12uc231 commented 1 year ago

Sounds good to me. Thanks for adding this!