Related debiasing methods to compare to

[stefanik12]

Prieskum related literatury na porovnanie nasej debiasing metody:

1. Don’t Take the Easy Way Out: popularny clanok, ktori si mnohi beru za referenciu pre porovnanie debiasingu aj na QA, napr nizsie uvedeny. Table 4 obsahuje porovnanie s inymi metodami, vratane baselinu = "reweighting" (trochu podobny nam)
2. Introspective Distillation for Robust Question Answering, Tables 9, 10: tento paper je pravdepodobne SOTA v robust QA trenovaneho na SQuADe, ale te implementacia je dost zlozita a pravpodeodbne v experimentalnom stave.

3. Look at the First Sentence Table 2 > BERT: tri rozne debiasing metody, v prislusnom repe slubuju, ze by mohli byt aj implementovane. Inak je to starsi paper, tj. nie je to idealny zdroj.

4. Mind the Trade-off: Debiasing NLU Models: Confidence Regularization metoda: Nieco ine ako Product-of-Experts. Downweighting pravdepodobnosti samplov, ktore su oznacene ako biased.

TODO1: zvazit este implementaciu a evaluaciu bias-agnostic metody:

5. Learning from Others' Mistakes: training mensieho "biased" modelu, nachylnejsieho na modelovanie biasov a regularizacia expected scores pri trainingu velkeho modelu. Podobne distilacii.

TODO2: Ak na to bude miesto, evaluovat este cross-bias performance: performance modelu trenovaneho proti jednemu biasu na inych biasoch.

[stefanik12]

Precital som si odkazovane papere, vsetky sa porovnavaju s LearnedMixinom (clanok 1, Sekcia 3 - 3.5.1), ktory je instanciou "Product of Experts" metody (v clanku 1 refered as BiasProduct, Sekcia 3.2.3).

Zaver:

1. Implementovat LearnedMixin (clanok 1)
2. Implementovat Confidence Regularization by the distillation of biased learners (clanok 4)