search

clab / lstm-parser

Transition-based dependency parser based on stack LSTMs
Apache License 2.0
204 stars 62 forks source link

GPU support and meaning of ppl and llh #24

Closed ghost closed 6 years ago

ghost commented 6 years ago

Hi,

I run the code, and it's already really fast. But I wonder if I can use GPU to speed up the training process? And if this is possible, what should I do?

And how to train the model without dynamic oracle?

By the way, what does 'llh xxx' and 'ppl xxx' means ?

[epoch=2 eta=0.0862069 clips=68 updates=100] update #325 (epoch 2.59906 |time=Fri Nov 10 19:35:08 2017 CST) llh: 704.437 ppl: 1.24006 err: 0.0610874 **dev (iter=325 epoch=2.59906) llh=0 ppl: 1 err: 1 uas: 0.86321 [2002 sents in 10265.1 ms]

miguelballesteros commented 6 years ago

Gpus are not going to give you significant speed ups since the parser runs with batch size 1 and given the nature of the problem it is very difficult to minibatch. You can try autobatching but you would have to change the code to allow several sentences at a time.

Ppl is perplexity Llh is log likelihood

ghost commented 6 years ago

Thanks for your reply :D

And can I run the code without dynamic oracle? According to README,

There is an easier to use version here, provided by duncanka, which can be used on data without oracle transitions.

But I found that oracle transitions are still needed there (actually I think it has an exactly the same README as here......),

java -jar ParserOracleArcStdWithSwap.jar -t -1 -l 1 -c training.conll > trainingOracle.txt java -jar ParserOracleArcStdWithSwap.jar -t -1 -l 1 -c development.conll > devOracle.txt

miguelballesteros commented 6 years ago

The fact that Oracle transitions are needed does not mean that it is not dynamic Oracle. I believe you are using the version without dynamic Oracle, though.

ghost commented 6 years ago

Oh, I had a wrong understanding....forgive my poor english.