eole-nlp / eole

Open language modeling toolkit based on PyTorch
https://eole-nlp.github.io/eole
MIT License
62 stars 12 forks source link

Update WMT17 recipe with working tokenization transforms examples #129

Closed francoishernandez closed 1 month ago

francoishernandez commented 1 month ago

Following #126 and other discsussions, it seems beneficial to add a bit more explicit examples for the tokenization transforms. The configs are not necessarily fully optimized, but do run and roughly reproduce the original results.

HURIMOZ commented 1 month ago

Thank you!