UniDT鲁棒优化算法 The implementation on Towards Scalable and Fast Distributionally Robust Optimization for Data-Driven Deep Learning.
MIT License
181
stars
26
forks
source link
readme
# WDRO（UniDT鲁棒优化算法）
The implementation on Towards Scalable and Fast Distributionally Robust Optimization for Data-Driven Deep Learning.
****
We formalize the learning problem with the supremum of a family of *subgroup-level* loss functions. Subgroup loss is the cost function of partitioned uncertainty set. Then we implement the maximum of subgroup loss as the objective function and update model parameters by reweighting the descent direction, calculated from a differentiable objective function:
***Prerequisites:***
```
tensorflow-gpu-1.13.1
cvxopt-1.2.5
numpy-1.15.0
pandas-0.25.2
```
#### Toydata:
A regression example. A toy dataset contains four subgroups.
```
python WDRO_for_toydata.py
```
#### Implemtation:
1, DNN+WDRO:
```
python 10-nonlinearized-dual.py --num_of_group 4 --num_minor 2 --train_subgroup_batch 100
```
Here,
--num_of_group : total subgroup number $N$ ;
--num_minor : number of minor-class subgroup $c$ ;
--train_subgroup_batch : number of data in each subgroup $s_j$ ;
2, ResNet+WDRO:
```
python nonlinearized-dual-ResNet.py --num_of_group 4 --num_minor 2 --train_subgroup_batch 100
```
The label now is "eyeglasses" in CelebA dataset, which can be changed in data_process.py.
## Acknowledgements
This work is supported by UniDT's Cognitive Computing and Few Shot Learning Project. This repository can only be used for personal/research/non-commercial purposes.
# WDRO（UniDT鲁棒优化算法）
The implementation on Towards Scalable and Fast Distributionally Robust Optimization for Data-Driven Deep Learning.
****
We formalize the learning problem with the supremum of a family of *subgroup-level* loss functions. Subgroup loss is the cost function of partitioned uncertainty set. Then we implement the maximum of subgroup loss as the objective function and update model parameters by reweighting the descent direction, calculated from a differentiable objective function:
***Prerequisites:***
```
tensorflow-gpu-1.13.1
cvxopt-1.2.5
numpy-1.15.0
pandas-0.25.2
```
#### Toydata:
A regression example. A toy dataset contains four subgroups.
```
python WDRO_for_toydata.py
```
#### Implemtation:
1, DNN+WDRO:
```
python 10-nonlinearized-dual.py --num_of_group 4 --num_minor 2 --train_subgroup_batch 100
```
Here,
--num_of_group : total subgroup number $N$ ;
--num_minor : number of minor-class subgroup $c$ ;
--train_subgroup_batch : number of data in each subgroup $s_j$ ;
2, ResNet+WDRO:
```
python nonlinearized-dual-ResNet.py --num_of_group 4 --num_minor 2 --train_subgroup_batch 100
```
The label now is "eyeglasses" in CelebA dataset, which can be changed in data_process.py.
## Acknowledgements
This work is supported by UniDT's Cognitive Computing and Few Shot Learning Project. This repository can only be used for personal/research/non-commercial purposes.