search

jeremy43 / Private_kNN

23 stars 9 forks source link

Full votes used instead of the clipped votes #2

Open adam-dziedzic opened 3 years ago

adam-dziedzic commented 3 years ago

Hello,

Thanks for sharing the code. It seems that despite setting config.use_tau to True, the full votes teachers_preds are used instead of the clipped votes preds_tau:

https://github.com/jeremy43/Private_kNN/blob/f321d0c32a4bd19a56fa488152403e0c62379bc3/face_attribute/knn_attribute.py#L174

jeremy43 commented 3 years ago

Thanks for spotting this issue! I have updated the corresponding code in the new commit. I tried to rerun the experiments on CelebA to compare the effects induced by tau (clipping threshold). The accuracy is now dropped to 84.70% (slightly worse than what I reported in the paper), when tau =10, k=800, sigma = 50, number of queries = 800 and sampling ratio = 0.05. If tau is set to be 5 and other hyper-parameters remain the same, the accuracy is around 84.0% while privacy cost epsilon <1.0. The feature is based on ImageNet pre-trained Resnet50m (using hog-based feature will drop the accuracy by around one percent). Hope this would be useful for you.

adam-dziedzic commented 3 years ago

Thanks for fixing it. I added a comment to the code: Shouldn't the teachers_preds be replaced with preds_tau as well for the count_zero_list?