search

AlignmentResearch / tuned-lens

Tools for understanding how transformer predictions are built layer-by-layer
https://tuned-lens.readthedocs.io/en/latest/
MIT License
438 stars 47 forks source link

Switch to using `simple-parsing` #64

Closed alexmlong closed 1 year ago

alexmlong commented 1 year ago

This moves the various CLI args specified in main into their appropriate files and puts them into a TypedDict structure to add typing.

norabelrose commented 1 year ago

We should probably just use https://github.com/lebrice/SimpleParsing like https://github.com/EleutherAI/elk does

alexmlong commented 1 year ago

We should probably just use https://github.com/lebrice/SimpleParsing like https://github.com/EleutherAI/elk does

Nice! Yeah that looks great. I just tested it out and it looks like a good option. I had one question I couldn't figure out about how to avoid nesting the args with the required dest keyword so I put a comment on their repo asking for help. I'll migrate as much as I can anyways. Thanks for the suggestion! 👍

alexmlong commented 1 year ago

Hi @norabelrose and @levmckinney , I've added simple-parsing but there are 2 things I can't figure out how to do with their API:

  1. If an argument is positional (e.g. model_name or dataset), it's not clear to me how to specify that in simple-parsing. It seems you just have to revert to using add_argument but then you lose the typing. Is that the right way to do it? If so, should I duplicate the typing in the dataclass?
  2. When you call add_arguments to add a dataclass it requires that you specify a dest which nests all the arguments under a certain field. Is there a way to prevent this that you know about? Otherwise I think I'll need to unwrap whatever the dest field is in each of the places where the args are used. Does that sound right?

Thanks

alexmlong commented 1 year ago

I figured some stuff out and am making changes, marking PR as draft for now until I'm ready for re-review