search

thu-spmi / CAT

A CRF-based ASR Toolkit
Apache License 2.0
324 stars 74 forks source link

new eroor on mc flatphone finetuning #61

Closed Sar-Dar closed 2 years ago

Sar-Dar commented 2 years ago

https://github.com/thu-spmi/CAT/blob/15ed6f22b31f76f77c1349d32b824b92b1667629/egs/commonvoice/run_mc.sh#L248-L274 when I run fine-tune stage, occurrd this error, could you help me find the solution?

'exp/mc_flatphone/scripts.tar.gz' already exists.
If you want to update it, please manually rm it then re-run this script.
Global number of GPUs: 1
Use GPU: local[0] | global[0]
> Data prepare
  Data prepared.

>>> Disable SpecAug <<<

[GPU 0]: Resuming from: exp/mc_flatphone/ckpt/bestckpt.pt
Traceback (most recent call last):
  File "ctc-crf/train.py", line 211, in <module>
    main_spawner(args, main_worker)
  File "ctc-crf/train.py", line 38, in main_spawner
    mp.spawn(_main_worker, nprocs=ngpus_per_node, args=(ngpus_per_node, args))
  File "/ssdhome/sardar321/anaconda3/envs/torch/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 230, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/ssdhome/sardar321/anaconda3/envs/torch/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 188, in start_processes
    while not context.join():
  File "/ssdhome/sardar321/anaconda3/envs/torch/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 150, in join
    raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException: 

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/ssdhome/sardar321/anaconda3/envs/torch/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 59, in _wrap
    fn(i, *args)
  File "/ssdhome/sardar321/CAT/egs/commonvoice/ctc-crf/train.py", line 94, in main_worker
    manager = coreutils.Manager(build_model, args)
  File "/ssdhome/sardar321/CAT/scripts/ctc-crf/coreutils.py", line 66, in __init__
    self.model, lr = update_model(
  File "/ssdhome/sardar321/CAT/scripts/ctc-crf/mc_lingual.py", line 85, in update_model
    src_idx, des_idx, pv, hdim, odim, mode, usg, lr = load_mc_conf(args)
  File "/ssdhome/sardar321/CAT/scripts/ctc-crf/mc_lingual.py", line 74, in load_mc_conf
    pv = load_pv(config["P"])
  File "/ssdhome/sardar321/CAT/scripts/ctc-crf/mc_lingual.py", line 61, in load_pv
    pv = np.load(fin)
  File "/ssdhome/sardar321/anaconda3/envs/torch/lib/python3.8/site-packages/numpy/lib/npyio.py", line 417, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
TypeError: expected str, bytes or os.PathLike object, not NoneType
Sar-Dar commented 2 years ago

It seems like the problem caused by parameter P in this config file: https://github.com/thu-spmi/CAT/blob/15ed6f22b31f76f77c1349d32b824b92b1667629/egs/commonvoice/conf/mc_flatphone_finetune_eval_de.json#L4 so how can I fix this?