Implementation of Kwon and Zou Data-OOB: Out-of-bag Estimate as a Sim…

[BastienZim]

…ple and Efficient Data Value ICML 2023 using pyDVL

Description

This PR adds the implementation of a data valuation method described in Kwon and Zou "Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Value" published at ICML 2023.

The notebook provided gives a comprehensive overview of the method, through examples, visualizations and point removal-evaluation.

No unit tests were added, as the notebook is testing the method. If-ever that is considered useful, I could write some.

Changes

• Created compute_data_oob which computed the data_oob data values. The function is compatible with all sklearn estimators as weak learners, provided through the utility model parameter.

Checklist

• [ ] Wrote Unit tests (if necessary)
• [x] Updated Documentation (if necessary)
• [ ] Updated Changelog
• [x] If notebooks were added/changed, added boilerplate cells are tagged with "tags": ["hide"] or "tags": ["hide-input"]

[BastienZim]

There still remains a problem raised by the type-checker that I did not manage to resolve. It concerns the return type of compute_data_oob. "Returning Any from function declared to return "ValuationResult[Any, Any]""

I would be interested in knowing the solution.

[AnesBenmerzoug]

Hi @BastienZim it looks good. It's almost ready there are just a few things missing:

• In order to render the notebook in the docs, you have to manually add nav entry to the mkdocs.yml file under nav -> Data Valuation -> Examples.
• In the notebook's introduction, you pasted the raw link to the publication. I think you should put that on the name of the paper just like you did in the docstring.
• Most matplotlib plotting functions return objects and they are printed out in jupyter notebooks. You should add a semi-colon ; at the end of the last line of a plotting call e.g. ax.set_xlabel("Point rank"); or assign the result to _ e.g. _ = plt.plot(np.arange(len(oob_values.values)), oob_values.values)
• We use tags on jupyter notebook cells to customize documentation rendering. We either hide an entire cell using the tag "hide" like import cells and the tag "hide-input" for plotting cells to hide the plotting code. Could you please add that? Refer to the Data Utility Learning notebook for an example.