search

microsoft / INMT-lite

Interactive Neural Machine Translation-lite (INMT-lite) is a framework to train and develop lite versions (.tflite) of models for neural machine translation (NMT) that can be run on embedded devices like mobile phones and tablets that have low computation power and space. The tflite models generated can be used to build the offline version of INMT mobile, a mobile version of INMT web.
MIT License
45 stars 7 forks source link

Missing files #32

Open Jalilnkh opened 1 year ago

Jalilnkh commented 1 year ago

You mentioned that for making the model Android-Compatible: We use an entirely different tokenization procedure. Could you let us know where are these files?

  1. Run final_tokenizer_train.py
  2. Run spm_extractor.py

I couldn't find them in GitHub.

harshitadd commented 1 year ago

Please navigate to the transformer-dev branch and use (a) _make_concatenatedvocab.py followed by (b) _spm_modelgenerator.py and finally (c) _spmextractor.py to train and generate the android compatible vocal files.

You can use the directory structure explanation to see the right description of each file for the transformer in case you notice any other files missing (possibly renamed and not adequately updated in the readme).