search

lucweytingh / ARL-UvA

A reproduced PyTorch implementation of the Adversarially Reweighted Learning (ARL) model, originally presented in "Fairness without Demographics through Adversarially Reweighted Learning".
MIT License
20 stars 3 forks source link

Bug in arl.py #1

Open hairfull opened 9 months ago

hairfull commented 9 months ago

Thanks for this pytorch-version implementation. However, I found a bug. In arl.py line 228-233:

      loss = F.binary_cross_entropy_with_logits(logits, targets)
      loss.to(self.device)
      weighted_loss = loss * adversary_weights
      weighted_loss = torch.mean(weighted_loss)
      return weighted_loss

However, in pytorch, F.binary_cross_entropy_with_logits() results in a scaler by default. As a result, the following multiply operators turns out meaningless, making this part work samely as the ERM model. loss = F.binary_cross_entropy_with_logits(logits, targets, reduction="none") should be correct.

MatteBelle commented 2 months ago

Hey! I've been trying to make this code work, could you share a working version if you have one?

I've been running hyperparameter.py and trying ways to fix it at the moment, but get this error:

tuning hyperparameters for law_school

--- (1/80) batch_size: 16, learner_lr: 0.001, adversary_lr: 0.001 /home/mattebelle/anaconda3/envs/arl_uva/lib/python3.8/site-packages/torch/cuda/init.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:100.) return torch._C._cuda_getDeviceCount() > 0 Training model 1/5 Traceback (most recent call last): File "/home/mattebelle/ARL-UvA/hyperparameter.py", line 186, in main() File "/home/mattebelle/ARL-UvA/hyperparameter.py", line 35, in main best_param, best_auc, step = optimize_parameters( File "/home/mattebelle/ARL-UvA/hyperparameter.py", line 111, in optimize_parameters metrics = train_for_n_iters( File "/home/mattebelle/ARL-UvA/train.py", line 229, in train_for_n_iters train_model( File "/home/mattebelle/ARL-UvA/train.py", line 117, in train_model auc_min, auc_macro_avg, auc_minority = metrics.set_auc_other(test_sigmoid, test_target, n_iters, test_dataset) File "/home/mattebelle/ARL-UvA/metrics.py", line 94, in set_auc_other for group in dataset.subgroup_indexes: AttributeError: 'IterableDataset' object has no attribute 'subgroup_indexes'"

lucweytingh commented 2 months ago

Dear Matteo, This error means you have not installed a Cuda driver. You should also consider installing the pytorch Cuda version to make things run on the GPU instead of the CPU. Check first if your system is capable of using Cuda. Met vriendelijke groet, kind regards,Luc WeytinghOn 17 Jun 2024, at 17:41, Matteo Belletti @.***> wrote: Hey! I've been trying to make this code work, could you share a working version if you have one? I've been getting this error when running hyperparameter.py, and trying ways to fix it at the moment " tuning hyperparameters for law_school --- (1/80) batch_size: 16, learner_lr: 0.001, adversary_lr: 0.001 /home/mattebelle/anaconda3/envs/arl_uva/lib/python3.8/site-packages/torch/cuda/init.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:100.) return torch._C._cuda_getDeviceCount() > 0 Training model 1/5 Traceback (most recent call last): File "/home/mattebelle/ARL-UvA/hyperparameter.py", line 186, in main() File "/home/mattebelle/ARL-UvA/hyperparameter.py", line 35, in main best_param, best_auc, step = optimize_parameters( File "/home/mattebelle/ARL-UvA/hyperparameter.py", line 111, in optimize_parameters metrics = train_for_n_iters( File "/home/mattebelle/ARL-UvA/train.py", line 229, in train_for_n_iters train_model( File "/home/mattebelle/ARL-UvA/train.py", line 117, in train_model auc_min, auc_macro_avg, auc_minority = metrics.set_auc_other(test_sigmoid, test_target, n_iters, test_dataset) File "/home/mattebelle/ARL-UvA/metrics.py", line 94, in set_auc_other for group in dataset.subgroup_indexes: AttributeError: 'IterableDataset' object has no attribute 'subgroup_indexes'"

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>