To train several open-source "canonical" vectors, starting from word2vec. The vectors will be made available publicly, to unlock vector/embedding based modelling with common NLP libraries such as SpaCy, as well as building of completely custom workflows with libraries such as TensorFlow.
There are several important distinctions here:
The body of text does not change; body of Tibetan Dharma texts is a constant
The vectors are made only for Dharma purpose i.e. the base of the vectors and the use are guaranteed to be the same!
We do not have to be too worried about downstream applications yet, we are simply making available the vectors as a fundamental resource in computational linguistics for Dharma
MATERIALS
More than ten thousand volumes of texts in OpenPecha
More than three thousand volumes of text in RinchenTerdzo
TOOLS
The fastest and the most robust way to get started is with Gensim, as it already has word2vec training built in. SpaCy recommends using Gensim for training word2vec vectors.
Core vectiorization workflow codes should be implementable to extend Signs.
OBJECTIVE
To train several open-source "canonical" vectors, starting from word2vec. The vectors will be made available publicly, to unlock vector/embedding based modelling with common NLP libraries such as SpaCy, as well as building of completely custom workflows with libraries such as TensorFlow.
There are several important distinctions here:
MATERIALS
TOOLS
The fastest and the most robust way to get started is with Gensim, as it already has word2vec training built in. SpaCy recommends using Gensim for training word2vec vectors.
Core vectiorization workflow codes should be implementable to extend Signs.