Given the continuous approach did not give us cool results (either did not change model performance or unlearned everything), we should try to see if sequential is any better.
(what epoch count, what lr?) Suggested: leave epoch count at 20, grab one of the “edge” LRs
Hypothesis:
Sequential unlearning unlearns faster with less samples.
Sequential unlearning is better at preserving desired capabilities while decreasing the targetted one.
Given the continuous approach did not give us cool results (either did not change model performance or unlearned everything), we should try to see if sequential is any better.
(what epoch count, what lr?) Suggested: leave epoch count at 20, grab one of the “edge” LRs
Hypothesis: