Improve DyNet usage - Githubissues

UppsalaNLP / uuparser

A transition-based parser for Universal Dependencies with BiLSTM word and character representations.

Apache License 2.0

80 stars 25 forks source link

Improve DyNet usage #23

Open LoicGrobol opened 4 years ago

LoicGrobol commented 4 years ago

We can take better advantage of what DyNet offers

Activating autobatching (easy to do and seems to improve speed in at least my anecdotal benchmark :-)
Remove noop dy.parameter calls

mdelhoneux commented 4 years ago

Just saw this, regarding autobatching, I tried to get it to work a while ago (2 years?) and accuracy took a hit :/ talked to Yoav Goldberg about it who did not seem surprised

LoicGrobol commented 4 years ago

Ah, too bad. Did he have an idea why?

mdelhoneux commented 4 years ago

Well batching is good for efficiency not accuracy so with a larger batch size, you need to train for more epochs if you want to reach the same result as with no batch at all. I didn't experiment so much so it might still be worth doing, especially if you want to train models on multiple treebanks, but you probably will need to train for more epochs and might not gain so much in training speed in the end