hicliff
commented
3 months ago 使用funasr里的镜像搭建环境，跑通了，可能跟我的环境有关。我的系统是CentOS Linux release 7.6.1810，我看镜像里的系统是Ubuntu 20.04.4 LTS
Open hicliff opened 3 months ago
hicliff
commented
3 months ago 使用funasr里的镜像搭建环境，跑通了，可能跟我的环境有关。我的系统是CentOS Linux release 7.6.1810，我看镜像里的系统是Ubuntu 20.04.4 LTS
bigchou
commented
3 months ago My system environment is: system: Ubuntu 20.04.6 LTS funasr.version == '1.0.16' pytorch version = 2.2.1
I also encountered this problem, but setting nj to 0 in run.sh allowed me to run CMVN successfully.
However, I then encountered other errors in stage 4:
[2024-03-17 01:39:37,527] torch.distributed.run: [WARNING]
[2024-03-17 01:39:37,527] torch.distributed.run: [WARNING]
[2024-03-17 01:39:37,527] torch.distributed.run: [WARNING] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
[2024-03-17 01:39:37,527] torch.distributed.run: [WARNING]
If you want to use the speaker diarization, please pip install hdbscan
If you want to use the speaker diarization, please pip install hdbscan
{'model': 'Paraformer', 'model_conf': {'ctc_weight': 0.3, 'lsm_weight': 0.1, 'length_normalized_loss': False, 'predictor_weight': 1.0, 'sampling_ratio': 0.4, 'use_1st_decoder_loss': True}, 'encoder': 'ConformerEncoder', 'encoder_conf': {'output_size': 256, 'attention_heads': 4, 'linear_units': 2048, 'num_blocks': 12, 'dropout_rate': 0.1, 'positional_dropout_rate': 0.1, 'attention_dropout_rate': 0.0, 'input_layer': 'conv2d', 'normalize_before': True, 'pos_enc_layer_type': 'rel_pos', 'selfattention_layer_type': 'rel_selfattn', 'activation_type': 'swish', 'macaron_style': True, 'use_cnn_module': True, 'cnn_module_kernel': 15}, 'decoder': 'ParaformerSANDecoder', 'decoder_conf': {'attention_heads': 4, 'linear_units': 2048, 'num_blocks': 6, 'dropout_rate': 0.1, 'positional_dropout_rate': 0.1, 'self_attention_dropout_rate': 0.0, 'src_attention_dropout_rate': 0.0}, 'predictor': 'CifPredictor', 'predictor_conf': {'idim': 256, 'threshold': 1.0, 'l_order': 1, 'r_order': 1, 'tail_threshold': 0.45}, 'frontend': 'WavFrontend', 'frontend_conf': {'fs': 16000, 'window': 'hamming', 'n_mels': 80, 'frame_length': 25, 'frame_shift': 10, 'lfr_m': 1, 'lfr_n': 1, 'cmvn_file': '../DATA/data/train/am.mvn'}, 'specaug': 'SpecAug', 'specaug_conf': {'apply_time_warp': True, 'time_warp_window': 5, 'time_warp_mode': 'bicubic', 'apply_freq_mask': True, 'freq_mask_width_range': [0, 30], 'num_freq_mask': 2, 'apply_time_mask': True, 'time_mask_width_range': [0, 40], 'num_time_mask': 2}, 'train_conf': {'accum_grad': 1, 'grad_clip': 5, 'max_epoch': 150, 'keep_nbest_models': 10, 'avg_nbest_model': 5, 'log_interval': 50}, 'optim': 'adam', 'optim_conf': {'lr': 0.0005}, 'scheduler': 'warmuplr', 'scheduler_conf': {'warmup_steps': 30000}, 'dataset': 'AudioDataset', 'dataset_conf': {'index_ds': 'IndexDSJsonl', 'batch_sampler': 'RankFullLocalShuffleBatchSampler', 'batch_type': 'example', 'batch_size': 32, 'max_token_length': 2048, 'buffer_size': 1024, 'shuffle': True, 'num_workers': 4, 'preprocessor_speech': 'SpeechPreprocessSpeedPerturb', 'preprocessor_speech_conf': {'speed_perturb': [0.9, 1.0, 1.1]}}, 'tokenizer': 'CharTokenizer', 'tokenizer_conf': {'unk_symbol': '
tables:
----------- dataset_classes -------------- | register name | class name | class location | | AudioDataset | AudioDataset | funasr/datasets/audio_datasets/datasets.py:8 | | AudioDatasetHotword | AudioDatasetHotword | funasr/datasets/audio_datasets/datasets.py:103 | | AudioLLMARDataset | AudioLLMARDataset | funasr/datasets/llm_datasets/datasets.py:273 | | AudioLLMDataset | AudioLLMDataset | funasr/datasets/llm_datasets/datasets.py:147 | | AudioLLMNARDataset | AudioLLMNARDataset | funasr/datasets/llm_datasets/datasets.py:8 | | AudioLLMQwenAudioDataset | AudioLLMQwenAudioDataset | funasr/datasets/llm_datasets_qwenaudio/datasets.py:8 | | AudioLLMVicunaDataset | AudioLLMVicunaDataset | funasr/datasets/llm_datasets_vicuna/datasets.py:8 | ----------- index_ds_classes -------------- | register name | class name | class location | | IndexDSJsonl | IndexDSJsonlRankFull | funasr/datasets/audio_datasets/index_ds.py:73 | | IndexDSJsonlRankFull | IndexDSJsonlRankFull | funasr/datasets/audio_datasets/index_ds.py:73 | | IndexDSJsonlRankSplit | IndexDSJsonlRankSplit | funasr/datasets/audio_datasets/index_ds.py:12 | ----------- preprocessor_classes -------------- | register name | class name | class location | | SpeechPreprocessSpeedPerturb | SpeechPreprocessSpeedPerturb | funasr/datasets/audio_datasets/preprocessor.py:18 | | TextPreprocessRemovePunctuation | TextPreprocessRemovePunctuation | funasr/datasets/llm_datasets/preprocessor.py:20 | | TextPreprocessSegDict | TextPreprocessSegDict | funasr/datasets/audio_datasets/preprocessor.py:38 | ----------- batch_sampler_classes -------------- | register name | class name | class location | | BatchSampler | RankFullLocalShuffleBatchSampler | funasr/datasets/llm_datasets/samplers.py:88 | | CustomDistributedBatchSampler | CustomDistributedBatchSampler | funasr/datasets/llm_datasets_vicuna/samplers.py:154 | | CustomDistributedBatchSampler_fn | CustomDistributedBatchSampler_fn | funasr/datasets/llm_datasets_vicuna/samplers.py:145 | | DistributedSamplerWarp | DistributedSamplerWarp | funasr/datasets/llm_datasets_vicuna/samplers.py:104 | | DynamicBatchLocalShuffleSampler | BatchSampler | funasr/datasets/llm_datasets/samplers.py:9 | | RankFullGlobalShuffleBatchSampler | RankFullGlobalShuffleBatchSampler | funasr/datasets/llm_datasets_vicuna/samplers.py:13 | | RankFullLocalShuffleBatchSampler | RankFullLocalShuffleBatchSampler | funasr/datasets/llm_datasets/samplers.py:88 | | RankFullLocalShuffleDynamicBatchSampler | RankFullLocalShuffleDynamicBatchSampler | funasr/datasets/llm_datasets/samplers.py:181 | ----------- frontend_classes -------------- | register name | class name | class location | | DefaultFrontend | DefaultFrontend | funasr/frontends/default.py:21 | | WavFrontend | WavFrontend | funasr/frontends/wav_frontend.py:78 | | WavFrontendOnline | WavFrontendOnline | funasr/frontends/wav_frontend.py:216 | | WhisperFrontend | WhisperFrontend | funasr/frontends/whisper_frontend.py:10 | ----------- joint_network_classes -------------- | register name | class name | class location | | joint_network | JointNetwork | funasr/models/transducer/joint_network.py:12 | ----------- model_classes -------------- | register name | class name | class location | | BAT | BAT | funasr/models/bat/model.py:35 | | BiCifParaformer | BiCifParaformer | funasr/models/bicif_paraformer/model.py:37 | | Branchformer | Branchformer | funasr/models/branchformer/model.py:6 | | CAMPPlus | CAMPPlus | funasr/models/campplus/model.py:30 | | CTTransformer | CTTransformer | funasr/models/ct_transformer/model.py:33 | | CTTransformerStreaming | CTTransformerStreaming | funasr/models/ct_transformer_streaming/model.py:27 | | Conformer | Conformer | funasr/models/conformer/model.py:8 | | ContextualParaformer | ContextualParaformer | funasr/models/contextual_paraformer/model.py:40 | | EBranchformer | EBranchformer | funasr/models/e_branchformer/model.py:6 | | Emotion2vec | Emotion2vec | funasr/models/emotion2vec/model.py:34 | | FsmnVADStreaming | FsmnVADStreaming | funasr/models/fsmn_vad_streaming/model.py:267 | | LCBNet | LCBNet | funasr/models/lcbnet/model.py:26 | | LLMASR | LLMASR | funasr/models/llm_asr/model.py:23 | | LLMASRNAR | LLMASRNAR | funasr/models/llm_asr_nar/model.py:24 | | LLMASRNARPrompt | LLMASRNARPrompt | funasr/models/llm_asr_nar/model.py:353 | | MonotonicAligner | MonotonicAligner | funasr/models/monotonic_aligner/model.py:24 | | OpenAIWhisperLIDModel | OpenAIWhisperLIDModel | funasr/models/whisper_lid/model.py:453 | | OpenAIWhisperModel | OpenAIWhisperModel | funasr/models/whisper_lid/model.py:21 | | Paraformer | Paraformer | funasr/models/paraformer/model.py:27 | | ParaformerStreaming | ParaformerStreaming | funasr/models/paraformer_streaming/model.py:37 | | Qwen-Audio | QwenAudioWarp | funasr/models/qwen_audio/model.py:17 | | Qwen-Audio-Chat | QwenAudioChatWarp | funasr/models/qwen_audio/model.py:72 | | Qwen/Qwen-Audio | QwenAudioWarp | funasr/models/qwen_audio/model.py:17 | | Qwen/Qwen-Audio-Chat | QwenAudioChatWarp | funasr/models/qwen_audio/model.py:72 | | Qwen/QwenAudio | QwenAudioWarp | funasr/models/qwen_audio/model.py:17 | | Qwen/QwenAudioChat | QwenAudioChatWarp | funasr/models/qwen_audio/model.py:72 | | QwenAudio | QwenAudioWarp | funasr/models/qwen_audio/model.py:17 | | QwenAudioChat | QwenAudioChatWarp | funasr/models/qwen_audio/model.py:72 | | QwenAudioChatWarp | QwenAudioChatWarp | funasr/models/qwen_audio/model.py:72 | | QwenAudioWarp | QwenAudioWarp | funasr/models/qwen_audio/model.py:17 | | SANM | SANM | funasr/models/sanm/model.py:13 | | SCAMA | SCAMA | funasr/models/scama/model.py:38 | | SeacoParaformer | SeacoParaformer | funasr/models/seaco_paraformer/model.py:43 | | Transducer | Transducer | funasr/models/transducer/model.py:34 | | Transformer | Transformer | funasr/models/transformer/model.py:20 | | UniASR | UniASR | funasr/models/uniasr/model.py:25 | | Whisper-base | WhisperWarp | funasr/models/whisper/model.py:16 | | Whisper-base.en | WhisperWarp | funasr/models/whisper/model.py:16 | | Whisper-large-v1 | WhisperWarp | funasr/models/whisper/model.py:16 | | Whisper-large-v2 | WhisperWarp | funasr/models/whisper/model.py:16 | | Whisper-large-v3 | WhisperWarp | funasr/models/whisper/model.py:16 | | Whisper-medium | WhisperWarp | funasr/models/whisper/model.py:16 | | Whisper-medium.en | WhisperWarp | funasr/models/whisper/model.py:16 | | Whisper-small | WhisperWarp | funasr/models/whisper/model.py:16 | | Whisper-small.en | WhisperWarp | funasr/models/whisper/model.py:16 | | Whisper-tiny | WhisperWarp | funasr/models/whisper/model.py:16 | | Whisper-tiny.en | WhisperWarp | funasr/models/whisper/model.py:16 | | WhisperWarp | WhisperWarp | funasr/models/whisper/model.py:16 | ----------- predictor_classes -------------- | register name | class name | class location | | CifPredictor | CifPredictor | funasr/models/paraformer/cif_predictor.py:15 | | CifPredictorV2 | CifPredictorV2 | funasr/models/paraformer/cif_predictor.py:143 | | CifPredictorV2Export | CifPredictorV2Export | funasr/models/paraformer/cif_predictor.py:378 | | CifPredictorV3 | CifPredictorV3 | funasr/models/bicif_paraformer/cif_predictor.py:95 | | CifPredictorV3Export | CifPredictorV3Export | funasr/models/bicif_paraformer/cif_predictor.py:340 | ----------- encoder_classes -------------- | register name | class name | class location | | BranchformerEncoder | BranchformerEncoder | funasr/models/branchformer/encoder.py:294 | | ChunkConformerEncoder | ConformerChunkEncoder | funasr/models/conformer/encoder.py:893 | | ConformerEncoder | ConformerEncoder | funasr/models/conformer/encoder.py:287 | | ConvBiasPredictor | ConvPredictor | funasr/models/lcbnet/encoder.py:358 | | DFSMN | DFSMN | funasr/models/fsmn_vad_streaming/encoder.py:318 | | EBranchformerEncoder | EBranchformerEncoder | funasr/models/e_branchformer/encoder.py:177 | | FSMN | FSMN | funasr/models/fsmn_vad_streaming/encoder.py:181 | | FSMNExport | FSMNExport | funasr/models/fsmn_vad_streaming/encoder.py:242 | | FusionSANEncoder | SelfSrcAttention | funasr/models/lcbnet/encoder.py:232 | | OpenAIWhisperEncoderWarp | OpenAIWhisperEncoderWarp | funasr/models/whisper_lid/encoder.py:17 | | QwenAudioEncoder | QwenAudioEncoder | funasr/models/qwen_audio/audio.py:338 | | RWKVEncoder | RWKVEncoder | funasr/models/rwkv_bat/rwkv_encoder.py:16 | | SANMEncoder | SANMEncoder | funasr/models/sanm/encoder.py:161 | | SANMEncoderChunkOpt | SANMEncoderChunkOpt | funasr/models/scama/encoder.py:162 | | SANMEncoderChunkOptExport | SANMEncoderExport | funasr/models/sanm/encoder.py:487 | | SANMEncoderExport | SANMEncoderExport | funasr/models/sanm/encoder.py:487 | | SANMVadEncoder | SANMVadEncoder | funasr/models/ct_transformer_streaming/encoder.py:148 | | SANMVadEncoderExport | SANMVadEncoderExport | funasr/models/ct_transformer_streaming/encoder.py:404 | | TransformerEncoder | TransformerEncoder | funasr/models/transformer/encoder.py:139 | | TransformerTextEncoder | TransformerTextEncoder | funasr/models/lcbnet/encoder.py:130 | ----------- decoder_classes -------------- | register name | class name | class location | | ContextualParaformerDecoder | ContextualParaformerDecoder | funasr/models/contextual_paraformer/decoder.py:103 | | ContextualParaformerDecoderExport | ContextualParaformerDecoderExport | funasr/models/contextual_paraformer/decoder.py:309 | | DynamicConvolution2DTransformerDecoder | DynamicConvolution2DTransformerDecoder | funasr/models/sa_asr/transformer_decoder.py:703 | | DynamicConvolutionTransformerDecoder | DynamicConvolutionTransformerDecoder | funasr/models/sa_asr/transformer_decoder.py:642 | | FsmnDecoder | FsmnDecoder | funasr/models/sanm/decoder.py:198 | | FsmnDecoderSCAMAOpt | FsmnDecoderSCAMAOpt | funasr/models/scama/decoder.py:197 | | LightweightConvolution2DTransformerDecoder | LightweightConvolution2DTransformerDecoder | funasr/models/sa_asr/transformer_decoder.py:580 | | LightweightConvolutionTransformerDecoder | LightweightConvolutionTransformerDecoder | funasr/models/sa_asr/transformer_decoder.py:519 | | OpenAIWhisperDecoderWarp | OpenAIWhisperDecoderWarp | funasr/models/whisper_lid/decoder.py:15 | | ParaformerDecoderSAN | ParaformerDecoderSAN | funasr/models/sa_asr/transformer_decoder.py:405 | | ParaformerDecoderSANExport | ParaformerDecoderSANExport | funasr/models/paraformer/decoder.py:1093 | | ParaformerSANDecoder | ParaformerSANDecoder | funasr/models/paraformer/decoder.py:979 | | ParaformerSANMDecoder | ParaformerSANMDecoder | funasr/models/paraformer/decoder.py:220 | | ParaformerSANMDecoderExport | ParaformerSANMDecoderExport | funasr/models/paraformer/decoder.py:637 | | ParaformerSANMDecoderOnlineExport | ParaformerSANMDecoderOnlineExport | funasr/models/paraformer/decoder.py:823 | | TransformerDecoder | TransformerDecoder | funasr/models/sa_asr/transformer_decoder.py:356 | | rnn_decoder | RNNDecoder | funasr/models/transducer/rnn_decoder.py:86 | | rnnt_decoder | RNNTDecoder | funasr/models/transducer/rnnt_decoder.py:14 | ----------- adaptor_classes -------------- | register name | class name | class location | | Linear | Linear | funasr/models/llm_asr_nar/adaptor.py:6 | | QFormer | EncoderProjectorQFormer | funasr/models/llm_asr/adaptor.py:31 | ----------- normalize_classes -------------- | register name | class name | class location | | GlobalMVN | GlobalMVN | funasr/models/normalize/global_mvn.py:11 | | UtteranceMVN | UtteranceMVN | funasr/models/normalize/utterance_mvn.py:8 | ----------- specaug_classes -------------- | register name | class name | class location | | SpecAug | SpecAug | funasr/models/specaug/specaug.py:14 | | SpecAugLFR | SpecAugLFR | funasr/models/specaug/specaug.py:104 | ----------- lid_predictor_classes -------------- | register name | class name | class location | | LidPredictor | LidPredictor | funasr/models/whisper_lid/lid_predictor.py:5 | ----------- tokenizer_classes -------------- | register name | class name | class location | | CharTokenizer | CharTokenizer | funasr/tokenizer/char_tokenizer.py:11 | | HuggingfaceTokenizer | HuggingfaceTokenizer | funasr/tokenizer/hf_tokenizer.py:10 | | SentencepiecesTokenizer | SentencepiecesTokenizer | funasr/tokenizer/sentencepiece_tokenizer.py:11 | | WhisperTokenizer | WhisperTokenizer | funasr/tokenizer/whisper_tokenizer.py:9 |
[2024-03-17 01:39:43,764][root][WARNING] - Using legacy_rel_pos and it will be deprecated in the future. [2024-03-17 01:39:43,776][root][INFO] - config.yaml is saved to: /alghome/timmy.wan/whisper/lab/VariousLargeWhisper/FunASR/examples/aishell/paraformer/exp/baseline_paraformer_conformer_12e_6d_2048_256_zh_char_exp1/config.yaml [2024-03-17 01:39:43,780][root][WARNING] - Using legacy_rel_pos and it will be deprecated in the future. [2024-03-17 01:39:43,800][root][WARNING] - Using legacy_rel_selfattn and it will be deprecated in the future. [2024-03-17 01:39:43,816][root][WARNING] - Using legacy_rel_selfattn and it will be deprecated in the future. No initialize method No initialize method [2024-03-17 01:39:44,941][root][INFO] - total_num of samplers across ranks: 120098 [2024-03-17 01:39:44,944][root][INFO] - total_num of samplers across ranks: 120098 [2024-03-17 01:39:44,996][root][INFO] - total_num of samplers across ranks: 14326 No checkpoint found at '/alghome/timmy.wan/whisper/lab/VariousLargeWhisper/FunASR/examples/aishell/paraformer/exp/baseline_paraformer_conformer_12e_6d_2048_256_zh_char_exp1/model.pt', does not resume status! [2024-03-17 01:39:45,000][root][INFO] - total_num of samplers across ranks: 14326 No checkpoint found at '/alghome/timmy.wan/whisper/lab/VariousLargeWhisper/FunASR/examples/aishell/paraformer/exp/baseline_paraformer_conformer_12e_6d_2048_256_zh_char_exp1/model.pt', does not resume status!
rank: 0, Training Epoch: 1: 0%|[34m [0m| 0/1877 [00:00<?, ?it/s] rank: 1, Training Epoch: 1: 0%|[34m [0m| 0/1877 [00:00<?, ?it/s]ERROR: Unexpected segmentation fault encountered in worker.
Notice: In order to resolve issues more efficiently, please raise issue following the template. （注意：为了更加高效率解决您遇到的问题，请按照模板提问，补充细节）
🐛 Bug
使用1.x版本funasr，跑aishell训练例子时，在stage 1 compute_audio_cmvn.py 步骤出现Segmentation fault
To Reproduce
Steps to reproduce the behavior (always include the command you ran):
Expected behavior
正确完成cmvn的计算
Environment
pip, source): 都试过，均报错Additional context