fasttext-langdetect/lid.176.bin, hugginface based models & argos translate models cached for good
workers completely offline vs hugginface
Not being able to read github once in a while (due to github being down or http errors), won't affect the worker like before.
More resilience, less dependencies to 3rd parties, road to decentralization.
WtpSplit bumped to 1.3.0 version, adding accuracy to detect languages from text.
SET huggingface transformers cache
ENV TRANSFORMERS_OFFLINE=1 ENV HF_DATASETS_OFFLINE=1 ENV HF_HUB_OFFLINE=1
fasttext-langdetect/lid.176.bin, hugginface based models & argos translate models cached for good workers completely offline vs hugginface Not being able to read github once in a while (due to github being down or http errors), won't affect the worker like before. More resilience, less dependencies to 3rd parties, road to decentralization.
WtpSplit bumped to 1.3.0 version, adding accuracy to detect languages from text.