PranjalChitale
commented
5 months ago usage: ct2-fairseq-converter [-h] --model_path MODEL_PATH --data_dir DATA_DIR [--user_dir USER_DIR]
[--fixed_dictionary FIXED_DICTIONARY] [--source_lang SOURCE_LANG] [--target_lang TARGET_LANG]
[--no_default_special_tokens] --output_dir OUTPUT_DIR [--vocab_mapping VOCAB_MAPPING]
[--quantization {int8,int8_float16,int16,float16}] [--force]As we are using a customized architecture, you need to use the --user_dir flag when using the converter and specify the path to model_configs directory present in your local clone.
2024-04-02 10:58:39.967723: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-04-02 10:58:39.967828: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-04-02 10:58:40.110310: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2024-04-02 10:58:43.291466: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-04-02 10:58:47 | INFO | fairseq.tasks.text_to_speech | Please install tensorboardX: pip install tensorboardX Traceback (most recent call last): File "/usr/local/bin/ct2-fairseq-converter", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/ctranslate2/converters/fairseq.py", line 341, in main
converter.convert_from_args(args)
File "/usr/local/lib/python3.10/dist-packages/ctranslate2/converters/converter.py", line 50, in convert_from_args
return self.convert(
File "/usr/local/lib/python3.10/dist-packages/ctranslate2/converters/converter.py", line 89, in convert
model_spec = self._load()
File "/usr/local/lib/python3.10/dist-packages/ctranslate2/converters/fairseq.py", line 166, in _load
spec = _get_model_spec(args)
File "/usr/local/lib/python3.10/dist-packages/ctranslate2/converters/fairseq.py", line 32, in _get_model_spec
model_name = fairseq.models.ARCH_MODEL_NAME_REGISTRY[args.arch]
KeyError: 'transformer_18_18'
i am getting this error while binarizing