ZJU-DIVER / OpenSV

An Open-Source Framework for Shapley-based value intended for data valuation.
https://zju-diver.github.io/OpenSV
Apache License 2.0
6 stars 0 forks source link

Solver "monte_carlo" of DataShapley and CSShapley doesn't act properly #4

Open iotang opened 3 months ago

iotang commented 3 months ago

They only return zero for all inputs.

import sys

sys.path.append("..")

from sklearn.datasets import *

data = load_iris()
X = data.data
y = data.target

X_train, X_valid, y_train, y_valid = X[:120], X[120:], y[:120], y[120:]

from opensv import DataShapley

shap = DataShapley()
shap.load(X_train, y_train, X_valid, y_valid)
shap.solve('monte_carlo')
print(shap.get_values())

Shapley values:

[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]

If change the solver:

shap.solve('truncated_mc')

Shapley values:

...
  9.59333333e-02  4.65333333e-02  7.85333333e-02  6.09333333e-02
  8.11333333e-02  5.23333333e-02  8.88666667e-02  8.02666667e-02
  6.28000000e-02  7.06666667e-02  8.90666667e-02  1.05400000e-01
  9.64000000e-02  5.01333333e-02  4.58666667e-02  7.20000000e-02]
iotang commented 3 months ago

Notes: bug here probably

if abs(final_acc - acc):
    break