allenai / deep_qa

A deep NLP library, based on Keras / tf, focused on question answering (but useful for other NLP too)
Apache License 2.0
404 stars 132 forks source link

Better parameter logging #300

Closed matt-gardner closed 7 years ago

matt-gardner commented 7 years ago

It'd be nice if every time a parameter was read, there was a logging statement. This is for reproducibility across code changes, particularly with default parameters. If the default changes in a version of code, the saved parameter file associated with a model won't give the same result. If instead we print out the actual value used for each parameter, default or not, we'd have a complete picture of what parameters were used.

I'd probably do this by adding a method to common.params, something like get_with_default, that logs the value that actually gets returned, with a common prefix. Then you can grep the output for these statements, and rebuild a json file that has all of the parameters, including default values.