I have a corpus that would mix up Cantonese and English. It is the only source of data of that specific domain, so I don't have another cleaner source for training.
我去咗Central London食飯。
In the example sentence above, I am only interested in Cantonese not English, and I don't want to include any English in that trained output. Is it possible to ignore all the English here? Can I replace them with an unknown token like this?
I have a corpus that would mix up Cantonese and English. It is the only source of data of that specific domain, so I don't have another cleaner source for training.
In the example sentence above, I am only interested in Cantonese not English, and I don't want to include any English in that trained output. Is it possible to ignore all the English here? Can I replace them with an unknown token like this?