search

mmschlk / shapiq

Shapley Interactions for Machine Learning
https://shapiq.readthedocs.io
MIT License
149 stars 10 forks source link

rethink index in run_benchmark() #171

Closed hbaniecki closed 2 months ago

hbaniecki commented 3 months ago

perhaps it can "override" the index in approximators

sv_approximators = [
    shapiq.PermutationSamplingSII(n=n_players, index="k-SII", random_state=0), 
    shapiq.SHAPIQ(n=n_players, random_state=0),
    shapiq.SVARMIQ(n=n_players, random_state=0), 
    shapiq.KernelSHAPIQ(n=n_players, random_state=0)
]
# run the benchmark with the chosen parameters
results = shapiq.games.benchmark.run_benchmark(
    index="k-SII",
    order=2,
    games=games,
    approximators=sv_approximators,
    save_path="benchmark_results.json",
    budget_steps=[500, 1000, 2000, 4000],
    n_jobs=8
)
mmschlk commented 2 months ago

This is not really possible. The benchmark has multiple ways of instantiating and using approximators. They can be specified via a str, class, or already instantiated object. For a more flexible usage (e.g. using approximators for different indices that they are intended for) we leave the logic as is: An index must always be specified.