about ablation study - Githubissues

askforalfred / alfred

ALFRED - A Benchmark for Interpreting Grounded Instructions for Everyday Tasks

MIT License

352 stars 77 forks source link

about ablation study #106

Closed Yanchuan99 closed 2 years ago

Yanchuan99 commented 2 years ago

Hi, Thanks for your great work firstly! I'm going to do the ablation study including input ablation and model ablation. How can I achieve that? I mean, which parameter 's value should I change? Thank you.

MohitShridhar commented 2 years ago

@Yanchuan99, you can do this with train_seq2seq.py arguments here.

No vision: set --vis_dropout=1.0
No language: set --lang_dropout=1.0
Goal-only: set --zero_instr
Instr-only: set --zero_goal

Yanchuan99 commented 2 years ago

Thanks a lot!

Yanchuan99 commented 2 years ago

And I'm wondering why default value of --vis_dropout is 0.3.

MohitShridhar commented 2 years ago

Just some regularization to prevent overfitting: https://cs231n.github.io/neural-networks-2/