Closed cvanpay closed 4 years ago
I built the env from git cloned to /home/user/ALICE within Linux rather than the mounted drive in Windows C and still receive the error.
SylNet completed
Traceback (most recent call last):
File "extract_basic_features.py", line 10, in <module>
with open(curdir + '/tmp_data/features/SylNet_out_files.txt') as csv_file:
FileNotFoundError: [Errno 2] No such file or directory: '/home/craig/ALICE/tmp_data/features/SylNet_out_files.txt'
paste: /home/craig/ALICE/tmp_data/features/SylNet_out.txt: No such file or directory
regress_ALUCs.py:10: UserWarning: genfromtxt: Empty input file: "/home/craig/ALICE/tmp_data/features/final_feats.txt"
F = genfromtxt(curdir + "/tmp_data/features/final_feats.txt", delimiter='\t')
Traceback (most recent call last):
File "regress_ALUCs.py", line 20, in <module>
words_est = np.dot(F,X_words)
File "<__array_function__ internals>", line 6, in dot
ValueError: shapes (0,2) and (5,) not aligned: 2 (dim 1) != 5 (dim 0)
paste: /home/craig/ALICE/tmp_data/features/SylNet_out_files.txt: No such file or directory
rm: cannot remove '/home/craig/ALICE/tmp_data/features/ALUCs_out_individual_tmp.txt': No such file or directory
getFinalEstimates.py:10: UserWarning: genfromtxt: Empty input file: "/home/craig/ALICE/tmp_data/features/ALUCs_out_individual.txt"
F = genfromtxt(curdir + "/tmp_data/features/ALUCs_out_individual.txt", delimiter='\t')
ALICE completed. Results written to /home/craig/ALICE/ALICE_output.txt and /home/craig/ALICE/diarization_output.rttm.
Hi! What kind of input files are you using? Can you also copy&paste the syntax you are using to call ALICE?
Hi, thanks for creating ALICE! Sorry for omitting that information.
I am using it on a folder "wavs" which includes 62 5 minute .wav files.
I am doing this with $ sh run_ALICE.sh wavs/
Found wav files.
Creating config for pyannote.
Done creating config for pyannote.
Extracting KCHI
Feature extractors based on "yaafe" are not available because "yaafelib" could not be found.
/home/craig//miniconda3/envs/ALICE/lib/python3.6/site-packages/pyannote/audio/applications/base.py:87: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
params_yml = yaml.load(fp)
Test set: 0it [00:00, ?it/s]
It finds the files and works through each feature extractor of CHI, MAL, KCHI, FEM, SPEECH, but when it is complete, the above errors occur.
By the way, ALICE works very well on my macOS machine! But my PC is much more powerful, and this is the error I get when trying to run it on Windows 10 with my Windows Subsystem Linux 2 shell.
Hmm, as you said, it could be a problem with Windows. We haven't actually done testing & debugging on that, but I forgot to add that it to the readme that ALICE has been so far confirmed to run on OSX and a couple of tested Linux variants.
Could you comment away the cleanup producedures at the end of run_ALICE.sh like this:
'''
echo "ALICE completed. Results written to $THISDIR/ALICE_output.txt and $THISDIR/diarization_output.rttm." '''
and then check whether /home/craig/ALICE/tmp_data/features/ folder exists at all after running run_ALICE.sh? Also, you could check if $THISDIR/tmp_data/short/ contains a list of .wav files, which should be the outputs from the diarization. There seems to be some kind of disconnect between diarizer writing to $THISDIR/tmp_data/short/ and SylNet trying to read from there.
Thanks for the suggestion!
Here's what happens after I do that. Note - I just ran it on a subset of 2 of the files to save time.
Found wav files.
Creating config for pyannote.
Done creating config for pyannote.
Extracting KCHI
Feature extractors based on "yaafe" are not available because "yaafelib" could not be found.
/home/craig/miniconda3/envs/ALICE/lib/python3.6/site-packages/pyannote/audio/applications/base.py:87: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
params_yml = yaml.load(fp)
Test set: 2it [01:02, 31.31s/it]
Test set: 2it [00:01, 1.74it/s]
Extracting CHI
Feature extractors based on "yaafe" are not available because "yaafelib" could not be found.
/home/craig/miniconda3/envs/ALICE/lib/python3.6/site-packages/pyannote/audio/applications/base.py:87: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
params_yml = yaml.load(fp)
Test set: 2it [01:02, 31.33s/it]
Test set: 2it [00:01, 1.64it/s]
Extracting MAL
Feature extractors based on "yaafe" are not available because "yaafelib" could not be found.
/home/craig/miniconda3/envs/ALICE/lib/python3.6/site-packages/pyannote/audio/applications/base.py:87: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
params_yml = yaml.load(fp)
Test set: 2it [01:01, 30.57s/it]
Test set: 2it [00:01, 1.77it/s]
Extracting FEM
Feature extractors based on "yaafe" are not available because "yaafelib" could not be found.
/home/craig/miniconda3/envs/ALICE/lib/python3.6/site-packages/pyannote/audio/applications/base.py:87: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
params_yml = yaml.load(fp)
Test set: 2it [01:00, 30.36s/it]
Test set: 2it [00:01, 1.54it/s]
Extracting SPEECH
Feature extractors based on "yaafe" are not available because "yaafelib" could not be found.
/home/craig/miniconda3/envs/ALICE/lib/python3.6/site-packages/pyannote/audio/applications/base.py:87: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
params_yml = yaml.load(fp)
Test set: 2it [01:05, 32.66s/it]
Test set: 2it [00:01, 1.71it/s]
SylNet completed
Traceback (most recent call last):
File "extract_basic_features.py", line 10, in <module>
with open(curdir + '/tmp_data/features/SylNet_out_files.txt') as csv_file:
FileNotFoundError: [Errno 2] No such file or directory: '/home/craig/ALICE/tmp_data/features/SylNet_out_files.txt' paste: /home/craig/ALICE/tmp_data/features/SylNet_out.txt: No such file or directory
regress_ALUCs.py:10: UserWarning: genfromtxt: Empty input file: "/home/craig/ALICE/tmp_data/features/final_feats.txt"
F = genfromtxt(curdir + "/tmp_data/features/final_feats.txt", delimiter='\t')
Traceback (most recent call last):
File "regress_ALUCs.py", line 20, in <module>
words_est = np.dot(F,X_words)
File "<__array_function__ internals>", line 6, in dot
ValueError: shapes (0,2) and (5,) not aligned: 2 (dim 1) != 5 (dim 0)
paste: /home/craig/ALICE/tmp_data/features/SylNet_out_files.txt: No such file or directory
rm: cannot remove '/home/craig/ALICE/tmp_data/features/ALUCs_out_individual_tmp.txt': No such file or directory
getFinalEstimates.py:10: UserWarning: genfromtxt: Empty input file: "/home/craig/ALICE/tmp_data/features/ALUCs_out_individual.txt"
F = genfromtxt(curdir + "/tmp_data/features/ALUCs_out_individual.txt", delimiter='\t')
ALICE completed. Results written to /home/craig/ALICE/ALICE_output.txt and /home/craig/ALICE/diarization_output.rttm.
That folder does exist - and Sylnet_Out_Files.txt exists.
/home/craig/ALICE/tmp_data/short/a2_00000100_00002630.wav
/home/craig/ALICE/tmp_data/short/a2_00005910_00054930.wav
/home/craig/ALICE/tmp_data/short/a2_00068340_00075160.wav
/home/craig/ALICE/tmp_data/short/a2_00075330_00108400.wav
/home/craig/ALICE/tmp_data/short/a2_00111390_00130040.wav
... plus more
Sylnet_Out.txt also exists.
1
8
2
2
5
2
1
1
1
... plus more
And
$THISDIR/tmp_data/short/ has the files listed in Sylnet_Out.txt
additionally, this is the second error again, if it helps:
/home/craig/miniconda3/envs/ALICE/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/craig/miniconda3/envs/ALICE/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/craig/miniconda3/envs/ALICE/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/craig/miniconda3/envs/ALICE/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/craig/miniconda3/envs/ALICE/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/craig/miniconda3/envs/ALICE/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
DATA LOADING DONE
2020-02-27 07:01:35.693803: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
/home/craig/ALICE/tmp_data/features/SylNet_out.txt
I spent a lot of time working on this, but I think it is just an artifact of WSL2 in beta. Perhaps I will try again when support has increased for it. I've tested and gotten the script running on MacOS and Manjaro (Arch Linux). Additionally, I got the cuda option working on Arch Linux and it works fantastically. Processes around 1.75 seconds per 5 minute segment (each feature) !
@cvanpay Thank you so much for looking into this! This is very useful! I also apologize that the response has been slow at our end. I still think we need to look into the Windows compatibility in a broader sense. For now, I will add a note to the readme that Linux + OSX support exists, while Windows is not fully tested.
@orasanen it was my pleasure, thank you for making this tool! No worries about communication, it was kind of me trying to fit a round peg in a square hole. I have tested it and it has worked on both Ubuntu and Manjaro, with --gpu and without! So you can add that to the list of Linux distros you've tested on. I will probably try again as WSL2 gets broad release (it's only open for Windows Insiders right now) and when it has access to GPU (even if I did get it working, it can't access GPU so it was really limited anyway). Works very well on CPU - was processing around 30 sec/5 min/feature, but really shines on GPU.
Using Ubuntu through WSL 2 on Windows Build 19041.
Ubuntu Ver.
Conda Ver.
Problem
SylNet works for each it, going through each feature extractor of CHI, MAL, KCHI, FEM, SPEECH, but when it is complete, this error occurs. Seems to be an error pointing to /tmp_data/features/SylNet_out_files.txt'. Could be specifically due to using WSL 2 and the PATH, but I'm not sure. Diarization output looks correct but no counts are given. Error 2 is probably related, as it couldn't find the out_files.
Error 1
Error 2
Environment