Closed JingL1014 closed 7 years ago
i have also tried running dependency parser only given pos tags, the result is correct. So i think there might be something wrong with the pos tagger.
This resembles an issue that has been discussed in an e-mail thread. Avoid models trained on UD 1.2, try UD 1.4 instead. The older releases of the treebank had word forms with vowel diacritics. Such a tagger cannot tag normal Arabic text, because without the diacritics all words are unknown.
Thanks for your quick response. I will train the model on UD1.4.
As @dan-zeman wrote, the issue is with vocalization -- UD 1.2 training data was fully vocalized, so the UD 1.2 models require full diacritics.
The UD 1.4 Arabic data are already not vocalized, so training using UD 1.4 improves the situation (i.e., it works for unvocalized input).
We are also implementing an Arabic normalization algoritm to UDPipe and will release UD 1.4 models using that algorithm, which will allow to parse vocalized, unvocalized or semi-vocalized inputs.
Leaving open until we release working Arabic model.
The UDPipe 1.1 and the CoNLL2017 Arabic model should work better -- they remove vocalization marks before tagging and parsing.
Hi, Thanks for developing this useful tool. Recently, i am trying to use UDpipe on Arabic documents. I have tried the pre-trained model for UD1.2 and i also trained the model on UD1.2 treebank myself. But the pos tags returned by the tool are always "X". As a result the dependency parser returned wrong output as well. Could you help me with this problem? thanks!![udpipe_arabic](https://cloud.githubusercontent.com/assets/11895289/23007065/5cdfbec6-f3cb-11e6-8a82-33c58873f45f.png)