search

emanjavacas / pie

A fully-fledge PyTorch package for Morphological Analysis, tailored to morphologically rich and historical languages.
MIT License
22 stars 10 forks source link

Fixes highway and adds (unrelated) an option for caching the dataset #57

Closed emanjavacas closed 4 years ago

emanjavacas commented 4 years ago

It could've been separate PRs but the fixes are kind of clean.

PonteIneptique commented 4 years ago

I don't know if you PR and want me to review ?

emanjavacas commented 4 years ago

Ye, feel free. It is mostly that I didn't want to push into master directly. To keep it clean :-)

PonteIneptique commented 4 years ago

I just wanted to check with you, I'd be happy to serve as reviewer here and there. You might have to add me to some kind of team for it to be easy (without writing rights).

Otherwise, looks good to me :)

emanjavacas commented 4 years ago

Well, this got bigger than I expected. I had some code changes in dev that I wanted to incorporate here as well. I think this PR superseeds the entire dev branch. If we merge this we can also delete dev (which has just been a mixing pot for all kinds of unrelated changes). From here on, we could work on single feature PRs instead of relying on dev (which I've overdone). Maybe you could test a couple of runs on this before we merge to test that it isn't breaking anything.

Apart from the transformer, it also ships some optimization code to run random search. It's added to train.py and all it needs is a opt.json file (see example opt.json attached to the PR) and the parameter n_iter. It will basically run the same config file n_iter times, every time resampling parameters specified in opt.json according the the specified distribution.

For instance:

"lr": {
    "opt": "truncnorm",
    "params": {
      "mu": 0.0025,
      "std": 0.002,
      "lower": 0.0001,
      "upper": 1
    }
  },

will result in a new "lr" being sampled according to a truncated normal centered around 0.0025 truncated to (0.0001, 1) with std 0.002.

Feel free to test this as well if you are interested!

PonteIneptique commented 4 years ago

I love all of it, and it looks good to me.

Could you provide a documentation for optimize for users ? This is pretty much raw as it is :)