How to solve this problem?

[junjie1003]

I want to reproduce the experiment, and I have trained a group classifier:

But when I try to find a threshold and save the predictions of group classifier according to the README, I met the problem as follow:

How to solve this problem? Thank you!

[sangwon-jung94]

Hi. I'm sorry for the late response.

If you want to set the 'sv' value as 1, you don't have to find a threshold and predictions of group classifier because there is not any samples without group labels. The annotated_idx files are saved only when the sv is less than 1. (please refer to the line 114 in datasets/celeba.py)

[junjie1003]

Hi, I set the sv to 0.25, ran the command python main_groupclf.py --date 20230621 --model resnet18_dropout --method scratch --dataset celeba --version groupclfval --sv 0.25, but got the result below: It seems that the problem lies in splitting the group-labeled and unlabeled data, could you please tell me how to solve it? Thank you very much!!!

[Kqp1227]

so am I

[sangwon-jung94]

Sorry, there was a typo in instructions. You should modify "groupclfval" to "groupclf_val". I modified our instructions in the README file. Thank you!

[junjie1003]

Thank you for your help. I tried using command python main.py --date 20230630 --model resnet18_dropout --method lbc --dataset celeba --version cgl --sv 0.25 to train a fair model using the LBC method, but I encountered the following issue: Does the code provide an implementation for LBC? Thank you!

[junjie1003]

I also encountered an issue when running FairHSIC:

Which file from the two mentioned below should be in the missing directory? Why is this error occurring?

[SanghyukChun]

@junjie1003 First issue: There is another typo in the README. Please use the reweighting option for using LBC.

Second issue: Could you provide the full command for FairHSIC?

[junjie1003]

Full command for FairHSIC is python main.py --date 20230630 --model resnet18_dropout --method fairhsic --dataset celeba --labelwise --version cgl --sv 0.25 --lamb 100

[SanghyukChun]

https://github.com/naver-ai/cgl_fairness/blob/main/data_handler/ssl_dataset.py#L173-L188

Please check this code for the exact path name. You should use a classifier that has a name ending with sv0.25_groupclf_val.pt.

The issue looks like the issue of the hard-coded epoch configuration: https://github.com/naver-ai/cgl_fairness/blob/ea9f1a2810a269ef5e5d09d672be47e7635c594a/data_handler/ssl_dataset.py#L175

@sangwon79 Could you check this issue?

[sangwon-jung94]

Yes, as @SanghyukChun mentioned, you can solve this issue by modifying the file path. (https://github.com/naver-ai/cgl_fairness/blob/main/data_handler/ssl_dataset.py#L173-L188)

It seems like you set the epoch and model as 50 and resnet18_dropout. You can change line 174 and 175 with the epoch and model type you use, or can fully comment out this part (line 173-188) and then put the filepath directly.

[junjie1003]

Thank you for your response. I have already made the changes to the model and epoch. My confusion actually lies in determining which of the following files is the one required by ssl_dataset.py:

1. ./trained_models/20230630/celeba/scratch/resnet18_seed0_epochs70_bs128_lr0.001_sv0.25_groupclf_val.pt which is generated by step 1 "Train a group classifier"
2. ../data/celeba/20230630/resnet18_seed0_epochs70_bs128_lr0.001_sv0.25_groupclf_val.pt which is generated by step 2 "Find a threshold and save the predictions of group classifier"

Both of these files have the same name and meet the requirements. Which one should I use for the missing file ./data/celeba/group_clf/resnet18_seed0_epochs70_bs128_lr0.001_sv0.25_groupclf_val.pt? Mainly, the confusion arises from the fact that the directory ./data/celeba/group_clf was not generated in the previous steps. Hence, I am a bit perplexed. Thank you for your helpful responses in clearing my doubts. I am hoping to use your work as a baseline in my own research.

[junjie1003]

Do you have the code for the evaluation part after model training? Could you please provide the evaluation code to obtain the result values for the charts mentioned in the paper, based on the fair model after fair-training? Thank you!!!