search

Hyperparticle / udify

A single model that parses Universal Dependencies across 75 languages. Given a sentence, jointly predicts part-of-speech tags, morphology tags, lemmas, and dependency trees.
https://arxiv.org/abs/1904.02099
MIT License
220 stars 56 forks source link

Scalar mix #3

Closed niless closed 4 years ago

niless commented 4 years ago

I was not able to use scalar mix option by changing combine_layers to mix from all. mix_embedding is set to 12. Is there anything else that need to change in the config file?

Hyperparticle commented 4 years ago

Currently, the scalar mix is computed outside the BERT embedder, one for each task. That is why all layers are returned initially. If having just one global mix instead of task specific mixes is what you want, you'll have to modify the code in the base model definition.