Closed akshayvijapur closed 4 years ago
Maybe you have to change idlak-egs/tts_tangle_arctic/s2/cmd.sh
to ignore CUDA
replacing --gpu 1
by --gpu 0
.
You have to add skip_cuda_check=true
in conf files as last line:
idlak-egs/tts_tangle_arctic/s2/conf/dur-nn-splice5.conf
idlak-egs/tts_tangle_arctic/s2/conf/full-nn-splice5.conf
idlak-egs/tts_tangle_arctic/s2/conf/pitch-nn-splice5.conf
I tried but no luck, Attached log for your reference.
Here is the output
paste-feats scp:/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/data/en/ga/slt/train/pitch_feats.scp scp:/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldata/en/ga/slt/train/feats.scp ark,scp:/tmp/dnn_feats/idlak/en/ga/slt/train_f0lbl_feats.ark,/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lblf0data/en/ga/slt/train/feats.scp
WARNING (paste-feats[5.5.964~2-cb7ee]:main():paste-feats.cc:137) Missing utt slt_a0001_282 from input scp:/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldata/en/ga/slt/train/feats.scp
WARNING (paste-feats[5.5.964~2-cb7ee]:main():paste-feats.cc:137) Missing utt slt_a0001_438 from input scp:/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldata/en/ga/slt/train/feats.scp
WARNING (paste-feats[5.5.964~2-cb7ee]:main():paste-feats.cc:137) Missing utt slt_a0001_439 from input scp:/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldata/en/ga/slt/train/feats.scp
WARNING (paste-feats[5.5.964~2-cb7ee]:main():paste-feats.cc:137) Missing utt slt_a0001_589 from input scp:/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldata/en/ga/slt/train/feats.scp
WARNING (paste-feats[5.5.964~2-cb7ee]:main():paste-feats.cc:137) Missing utt slt_b0001_167 from input scp:/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldata/en/ga/slt/train/feats.scp
WARNING (paste-feats[5.5.964~2-cb7ee]:main():paste-feats.cc:137) Missing utt slt_b0001_391 from input scp:/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldata/en/ga/slt/train/feats.scp
WARNING (paste-feats[5.5.964~2-cb7ee]:main():paste-feats.cc:137) Missing utt slt_b0001_536 from input scp:/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldata/en/ga/slt/train/feats.scp
LOG (paste-feats[5.5.964~2-cb7ee]:main():paste-feats.cc:158) Done 1025 utts, errors on 7
paste-feats scp:/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/data/en/ga/slt/dev/pitch_feats.scp scp:/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldata/en/ga/slt/dev/feats.scp ark,scp:/tmp/dnn_feats/idlak/en/ga/slt/dev_f0lbl_feats.ark,/home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lblf0data/en/ga/slt/dev/feats.scp
LOG (paste-feats[5.5.964~2-cb7ee]:main():paste-feats.cc:158) Done 100 utts, errors on 0
##### Step 4: training DNNs #####
### Step 4a: duration model DNN ###
run.pl: job failed, log is in /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/exp-dnn/en/ga/slt/tts_dnn_dur_3_delta_quin5/_train_nnet.log
Could you attach the file /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/exp-dnn/en/ga/slt/tts_dnn_dur_3_delta_quin5/_train_nnet.log
.
Or send the last 30 lines of the file.
attached, Please check https://github.com/Idlak/idlak/issues/127#issuecomment-621041837
The only difference with mine is lines 57 and 63, your log file has:
LOG (nnet-forward[5.5.964~2-cb7ee]:SelectGpuId():cu-device.cc:128) Manually selected to compute on CPU.
Did you select an option for CPU ? Did you compile Kaldi with cuda disabled:
cd kaldi/src
./configure --static --use-cuda=no
My log file: _train_nnet.log.zip
@lpierron Thank you, It is working now. Please add this to documentation.
Hi,
I have installed kaldi by following steps in the doc. I tried to run examples present https://github.com/Idlak/idlak/tree/master/idlak-egs/tts_tangle_arctic/s2
Seems like i am getting following error. Can you please guide me?
steps/train_nnet_basic.sh --config conf/dur-nn-splice5.conf /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldurdata/en/ga/slt/train /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldurdata/en/ga/slt/dev /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/durdata/en/ga/slt/train /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/durdata/en/ga/slt/dev /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/exp-dnn/en/ga/slt/tts_dnn_dur_3_delta_quin5
Started at Wed Apr 22 10:18:08 IST 2020
# steps/train_nnet_basic.sh --config conf/dur-nn-splice5.conf /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldurdata/en/ga/slt/train /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldurdata/en/ga/slt/dev /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/durdata/en/ga/slt/train /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/durdata/en/ga/slt/dev /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/exp-dnn/en/ga/slt/tts_dnn_dur_3_delta_quin5
INFO
steps/train_nnet_basic.sh : Training Neural Network dir : /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/exp-dnn/en/ga/slt/tts_dnn_dur_3_delta_quin5 Train-set : /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldurdata/en/ga/slt/train 1025, /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/durdata/en/ga/slt/train CV-set : /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/lbldurdata/en/ga/slt/dev 100 /home/avijapur/delete/deepspeachmodel/openvinokaldi/idlak/idlak-egs/tts_tangle_arctic/s2/durdata/en/ga/slt/dev
LOG ([5.5.964~2-cb7ee]:main():cuda-gpu-available.cc:49)
IS CUDA GPU AVAILABLE? 'avijapur-H370M-D3H'
ERROR ([5.5.964~2-cb7ee]:SelectGpuId():cu-device.cc:141) No CUDA GPU detected!, diagnostics: cudaError_t 100 : "no CUDA-capable device is detected", in cu-device.cc:141
[ Stack-Trace: ] kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const*) kaldi::FatalMessageLogger::~FatalMessageLogger() kaldi::CuDevice::SelectGpuId(std::cxx11::basic_string<char, std::char_traits, std::allocator >)
main
libc_start_main
_start
ERROR ([5.5.964~2-cb7ee]:SelectGpuId():cu-device.cc:141) No CUDA GPU detected!, diagnostics: cudaError_t 100 : "no CUDA-capable device is detected", in cu-device.cc:141
[ Stack-Trace: ] kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const*) kaldi::MessageLogger::~MessageLogger() kaldi::FatalMessageLogger::~FatalMessageLogger()