Our example collection is kind of messy and hard to navigate. This PR aims to make the organized and add github action to test them to enable long-term usability.
This PR does not add any examples. New example scripts will be updated according to the necessity and request.
2. Summary of the change
Rename the folder /example -> /examples
Reorganize current examples into three catagories (noisy label detection, use pretrained benchmark, and estimate brittleness). See here for a detailed guide.
Fix all 6 examples to ensure that both CPU and GPU users could run through them in a reasonable time (<2min), delete some redundant examples.
Add a github action to run the examples for each PR, so that broken examples will be found out timely. The github action will only check the completion of example running. Accuracy monitoring will be added in later PRs.
Some minor changes in the dattri repo (add cifar10 support, AttributionTask will load the checkpoints in the same device as the model, auc will transform the score to cpu to avoid user redundant code).
3. What tests have been added/updated for the change?
[x] Unit test: Typically, this should be included if you implemented a new function/fixed a bug.
[x] Application test: If you wrote an example for the toolkit, this test should be added.
Description
1. Motivation and Context
Our example collection is kind of messy and hard to navigate. This PR aims to make the organized and add github action to test them to enable long-term usability.
This PR does not add any examples. New example scripts will be updated according to the necessity and request.
2. Summary of the change
/example->/examplesdattrirepo (add cifar10 support,AttributionTaskwill load the checkpoints in the same device as the model, auc will transform the score to cpu to avoid user redundant code).3. What tests have been added/updated for the change?