search

ictnlp / StreamSpeech

StreamSpeech is an “All in One” seamless model for offline and simultaneous speech recognition, speech translation and speech synthesis.
https://ictnlp.github.io/StreamSpeech-site/
MIT License
683 stars 49 forks source link

Unable to run test script on local machine #1

Open maxjkwang opened 3 weeks ago

maxjkwang commented 3 weeks ago

Hi Mr. Zhang,

I was trying to recreate the results on my local machine, but I encountered the following error:

Traceback (most recent call last): File "/Users/ifrit/Library/Python/3.9/bin/simuleval", line 33, in sys.exit(load_entry_point('simuleval', 'console_scripts', 'simuleval')()) File "\<my streamspeech directory>/SimulEval/simuleval/cli.py", line 47, in main system, args = build_system_args() File "\<my streamspeech directory>/SimulEval/simuleval/utils/agent.py", line 131, in build_system_args system = system_class.from_args(args) File "\<my streamspeech directory>/SimulEval/simuleval/agents/agent.py", line 161, in from_args return cls(args) File "\<my streamspeech directory>/agent/speech_to_speech.streamspeech.agent.py", line 117, in init self.load_model_vocab(args) File "\<my streamspeech directory>/agent/speech_to_speech.streamspeech.agent.py", line 356, in load_model_vocab utils.import_user_module(state["cfg"].common) File "\<my streamspeech directory>/fairseq/fairseq/utils.py", line 481, in import_user_module raise FileNotFoundError(module_path) FileNotFoundError: /data/zhangshaolei/SimulS2S/reasearchs/ctc_unity

I changed all the config files as necessary, but this issue remains. I looked into the code a bit, and I believe it's due to the state of the loaded checkpoint file:

> print(state['cfg'].common.user_dir) - /data/zhangshaolei/SimulS2S/reasearchs/ctc_unity

My solution is to add a line changing the user_dir before the import_user_module call in line 356 in speech_to_speech.streamspeech.agent.py, and it's working fine for me now.

    state = checkpoint_utils.load_checkpoint_to_cpu(filename)
    state["cfg"].common.user_dir = 'reasearchs/ctc_unity/'
    utils.import_user_module(state["cfg"].common)

I'm not sure if it's due to me missing any config changes, but I think it's unlikely as it seems written into the pretrained weights. I've only tested Simultaneous S2ST on fr-en, but I guess similar errors will happen in other scenarios.

Thank you so much!

zhangshaolei1998 commented 3 weeks ago

Thanks for your reminder, it is possible that some paths are saved in the model. We will fix this issue soon.