tune

[jwijffels]

I was just training a udpipe lemmatiser myself on dutch and browsed a bit github for udpipe_train and saw this. I'm the author of the udpipe R package and happen to work on some 18th-19th century texts myself (dutch / french). Let me give some advice to make the udpipe model better, in order of importance

• You really should split your data in a train/dev dataset at https://github.com/CristinaGHolgado/Lemmatisation_AncienFran-ais/blob/master/Train_dataset/train_udpipe.R#L7 otherwise overtraining will be what you are doing
• make a separate model for the parts of speech tagger and the lemmatiser
• and tune the relevant parameters a bit - examples of parameters for medieval tagger on the srcmf corpus at https://github.com/bnosac/udpipe/blob/master/inst/models-ud-2.5/params_tagger#L32, for a lemmatiser, you could look at https://github.com/bnosac/udpipe/blob/master/inst/models-ud-2.5/params_tagger#L28 Hope this helps improving a bit :)

[CristinaGHolgado]

Hi @jwijffels That is very helpful to me as I am in the process of improving the lemmatization with this tool and Pie. I made a separate model for the tagger and the lemmatiser although it is not yet in the repository. There are certainly some steps to be improved, starting with the split in a dev set. Many thanks for your advices !