Closed emanjavacas closed 4 years ago
I don't know if you PR and want me to review ?
Ye, feel free. It is mostly that I didn't want to push into master directly. To keep it clean :-)
I just wanted to check with you, I'd be happy to serve as reviewer here and there. You might have to add me to some kind of team for it to be easy (without writing rights).
Otherwise, looks good to me :)
Well, this got bigger than I expected. I had some code changes in dev
that I wanted to incorporate here as well. I think this PR superseeds the entire dev
branch. If we merge this we can also delete dev
(which has just been a mixing pot for all kinds of unrelated changes). From here on, we could work on single feature PRs instead of relying on dev
(which I've overdone). Maybe you could test a couple of runs on this before we merge to test that it isn't breaking anything.
Apart from the transformer, it also ships some optimization code to run random search. It's added to train.py and all it needs is a opt.json file (see example opt.json attached to the PR) and the parameter n_iter
. It will basically run the same config file n_iter
times, every time resampling parameters specified in opt.json
according the the specified distribution.
For instance:
"lr": {
"opt": "truncnorm",
"params": {
"mu": 0.0025,
"std": 0.002,
"lower": 0.0001,
"upper": 1
}
},
will result in a new "lr" being sampled according to a truncated normal centered around 0.0025 truncated to (0.0001, 1) with std 0.002.
Feel free to test this as well if you are interested!
I love all of it, and it looks good to me.
Could you provide a documentation for optimize for users ? This is pretty much raw as it is :)
It could've been separate PRs but the fixes are kind of clean.