search

unlearning-challenge / starting-kit

Starting kit for the NeurIPS 2023 unlearning challenge
https://unlearning-challenge.github.io/
Apache License 2.0
373 stars 134 forks source link

How are the unlearning evaluation metrics computed? #17

Open saptab opened 6 months ago

saptab commented 6 months ago

Is the evaluation metric public?

Please share how the evaluation metric is computed

fabianp commented 6 months ago

We're working on making it open source. I expect it to take a couple of weeks

saptab commented 5 months ago

Hi Fabian, following up about the open source metric since its been two weeks :)

looking forward to learn about it and run it

fabianp commented 5 months ago

Unfortunately many people in the team have been busy with the ICML deadline, which I hadn't considered when I responded earlier. A more realistic timeline will be before the end of February. Apologies for the delay

On Sun, Feb 4, 2024 at 6:47 PM saptab @.***> wrote:

Hi Fabian, following up about the open source metric since its been two weeks :)

looking forward to learn about it and run it

— Reply to this email directly, view it on GitHub https://github.com/unlearning-challenge/starting-kit/issues/17#issuecomment-1925878766, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACDZB5MZ5TV7RKQ65KRQTTYR7JSVAVCNFSM6AAAAABCDTTQT6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRVHA3TQNZWGY . You are receiving this because you commented.Message ID: @.***>

yasser-h-khalil commented 4 months ago

Hi @fabianp, when will the evaluation metrics be released?

fugokidi commented 2 months ago

Hi @fabianp Any news on evaluation metrics?

fabianp commented 2 months ago

it's finally online: https://github.com/google-deepmind/unlearning_evaluation