search

unlearning-challenge / starting-kit

Starting kit for the NeurIPS 2023 unlearning challenge
https://unlearning-challenge.github.io/
Apache License 2.0
379 stars 133 forks source link

How are the unlearning evaluation metrics computed? #17

Open saptab opened 10 months ago

saptab commented 10 months ago

Is the evaluation metric public?

Please share how the evaluation metric is computed

fabianp commented 10 months ago

We're working on making it open source. I expect it to take a couple of weeks

saptab commented 9 months ago

Hi Fabian, following up about the open source metric since its been two weeks :)

looking forward to learn about it and run it

fabianp commented 9 months ago

Unfortunately many people in the team have been busy with the ICML deadline, which I hadn't considered when I responded earlier. A more realistic timeline will be before the end of February. Apologies for the delay

On Sun, Feb 4, 2024 at 6:47 PM saptab @.***> wrote:

Hi Fabian, following up about the open source metric since its been two weeks :)

looking forward to learn about it and run it

— Reply to this email directly, view it on GitHub https://github.com/unlearning-challenge/starting-kit/issues/17#issuecomment-1925878766, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACDZB5MZ5TV7RKQ65KRQTTYR7JSVAVCNFSM6AAAAABCDTTQT6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRVHA3TQNZWGY . You are receiving this because you commented.Message ID: @.***>

yasser-h-khalil commented 8 months ago

Hi @fabianp, when will the evaluation metrics be released?

fugokidi commented 7 months ago

Hi @fabianp Any news on evaluation metrics?

fabianp commented 6 months ago

it's finally online: https://github.com/google-deepmind/unlearning_evaluation