search

allenai / amti

A Mechanical Turk Interface (amti) 🤖
Apache License 2.0
55 stars 18 forks source link

Add CLIs for managing workers. #11

Closed rob-dalton closed 5 years ago

rob-dalton commented 5 years ago

Added commands for:

Added new utils file for worker management functions.

rob-dalton commented 5 years ago

For UI changes - we could break out commands into groups. For example:

I think click allows you to group commands pretty easily.

rob-dalton commented 5 years ago

Also I can't think of a better way to handle single vs multiple WorkerIds. It's a bit clunky, but I feel like you should be able to handle them with the same command, since it's the same action.

This way, you can provide any number of WorkerIds on the command line (makes it easy to handle one or two workers). And if you want to do a large batch, you can leave the IDS arg empty and just provide a file path.

nalourie-ai2 commented 5 years ago

Sounds good to me.

I really like the idea of having amti batch and amti workers command groups, but I think we could handle that in a follow up PR to keep things moving. Changing the UI like that might also justify a larger refactor of the code base, with subpackages for the different command groups.

Having one command per action makes sense to me UI-wise. We should handle the additional columns for other arguments in the case that it takes CSV input (like --reason for example). Similarly, we should allow the delimiter to be specified by the user, since at a minimum people will probably have both CSV and TSV files. Also, with two input modes, we should have more thorough validation, i.e.

  1. if --file is passed, we should check that ids is None and --reason is not used.
  2. if ids is passed, we should validate that --file is not passed and --reason is present.

Or something like that. This issue discusses implementing mutually exclusive options in click, though I think the simplest and probably best approach is to just put a little validation logic at the beginning of the command rather than trying to fit it into click's callbacks / parameter validation.