facebookresearch / ParlAI

A framework for training and evaluating AI models on a variety of openly available dialogue datasets.
https://parl.ai
MIT License
10.48k stars 2.1k forks source link

ParlAI cannot parse arguments for task-specific mutators. #4321

Closed zhangmozhi closed 2 years ago

zhangmozhi commented 2 years ago

Bug description When using task-specific mutator, mutator-specific arguments cannot be parsed.

Reproduction steps parlai display_data --task dailydialog:mutators=episode_shuffle --preserve-context true

Expected behavior There should be no error.

Logs

Parse Error: unrecognized arguments: --preserve-context true
stephenroller commented 2 years ago

Unfortunately, mutator args don't work when specified via the --task X:mutators= format. We should update the mutator docs to reflect this.

They will still parse if you specify them as root items (--task X --mutators episode_shuffle). You should also be able to force the opt in the task argument (--task X:mutators=episode_shuffle:preserve_context=True)

zhangmozhi commented 2 years ago

Unfortunately, mutator args don't work when specified via the --task X:mutators= format. We should update the mutator docs to reflect this.

They will still parse if you specify them as root items (--task X --mutators episode_shuffle). You should also be able to force the opt in the task argument (--task X:mutators=episode_shuffle:preserve_context=True)

The last option works. Thanks for the help!

stephenroller commented 2 years ago

Would you be open to adding this info on the mutator docs?

zhangmozhi commented 2 years ago

Will do