search

yonseivnl / sdp

Official Implementation of SDP (Scheduled Data Prior) (ICLR 2023)
7 stars 1 forks source link

Weird knowledge loss and gain rates #1

Open EvelynChee opened 5 months ago

EvelynChee commented 5 months ago

Hi,

Thanks for sharing your great work.

I am currently having some issue with the knowledge loss and gain rates when running the code. I tried the SDP method with the settings

DATASET="cifar100" 
SIGMA=10
REPEAT=1
INIT_CLS=100
SDP_MEAN=10000
SDP_VAR=0.75

The final results I have obtained are:

[INFO] ======== Summary =======
[INFO] A_auc 0.46821259412578764 | A_last 0.5561 | KLR_avg 0.7233384646848867 | KGR_avg 0.01556362787537161

The A_auc and A_last looks ok but the KLR_avg and KGR_avg values seem to differ a lot from what was provided in the paper. Am I missing something to reproduce the results?

Thanks.

Akila-Ayanthi commented 1 month ago

Hi, I also got the same results. Were you able to figure out if you've missed something?

EvelynChee commented 4 weeks ago

Hi @Akila-Ayanthi, I have yet to figure out what is the issue