benob / recasepunc

Model for recasing and repunctuating ASR transcripts
BSD 3-Clause "New" or "Revised" License
129 stars 20 forks source link

Russian model doesn't work, while English does #16

Open xenia19 opened 1 year ago

xenia19 commented 1 year ago

When I use Russian model, it gives me this error:

WARNING: reverting to cpu as cuda is not available
Some weights of the model checkpoint at DeepPavlov/rubert-base-cased were not used when initializing BertModel: ['cls.seq_relationship.bias', 'cls.predictions.decoder.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.bias', 'cls.seq_relationship.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight']

 File "C:\pypy\rus\recasepunc.py", line 741, in <module>
    main(config, config.action, config.action_args)
  File "C:\pypy\rus\recasepunc.py", line 715, in main
    generate_predictions(config, *args)
  File "C:\pypy\rus\recasepunc.py", line 349, in generate_predictions
    for line in sys.stdin:
  File "C:\Users\Xenia\AppData\Local\Programs\Python\Python39\lib\codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xef in position 0: invalid continuation byte

 File "C:\Users\Xenia\AppData\Local\Programs\Python\Python39\lib\site-packages\flask\app.py", line 1796, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)      
  File "C:\pypy\app.py", line 32, in process_audio
    cased = subprocess.check_output('python rus/recasepunc.py predict rus/checkpoint', shell=True, text=True, input=text)
  File "C:\Users\Xenia\AppData\Local\Programs\Python\Python39\lib\subprocess.py", 
line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "C:\Users\Xenia\AppData\Local\Programs\Python\Python39\lib\subprocess.py", 
line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'python rus/recasepunc.py predict rus/checkpoint' returned non-zero exit status 1.

Sorry for a long message, I'm not sure which of these messages are the most important. Should I use another version of transformers? I use transformers==4.16.2 and it works fine with English model.

jackfs1919 commented 4 months ago

добрый ден, у меня такое же, не нашла решение?