CMakeFiles/Makefile2:948: recipe for target 'CMakeFiles/fl_lib_audio.dir/all' failed

[benam2]

Im trying to install Flashlight to get a speech recognition project run. Its been really 2 weeks now!!! I have made mistake before and cleaned up everything and started from scratch. Now I have installed the flashlight and then when I want to install the python bindings after nearly 60 percent of the building going it raises a VERY WEIRD ERROR that I really don't know how to search the error since there is little to share.

Reproduction Steps

[Steps to reproduce. Please include, if needed for your issue:

• Any configurations or custom setup
• Commands to run: python3 setup.py install

This is the error part:

-- MKL library found
-- CBLAS found (include: /opt/intel/oneapi/mkl/latest/include, library: /opt/intel/oneapi/mkl/latest/lib/intel64/libmkl_intel_lp64.so;/opt/intel/oneapi/mkl/latest/lib/intel64/libmkl_gnu_thread.so;/opt/intel/oneapi/mkl/latest/lib/intel64/libmkl_core.so;-fopenmp;/usr/lib/x86_64-linux-gnu/libpthread.so;/usr/lib/x86_64-linux-gnu/libm.so;/usr/lib/x86_64-linux-gnu/libdl.so)
-- FFTW found
-- Found OpenMP_C: -fopenmp  
-- Found OpenMP_CXX: -fopenmp  
-- Found OpenMP: TRUE   
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ai-labs/flashlight/bindings/python/build/temp.linux-x86_64-3.7
make[2]: *** No rule to make target '/home/ai-labs/fftw-3.3.10/build/libfftw3.so.3', needed by '../lib.linux-x86_64-3.7/libfl_lib_audio.so.0.3.2'.  Stop.
CMakeFiles/Makefile2:948: recipe for target 'CMakeFiles/fl_lib_audio.dir/all' failed
make[1]: *** [CMakeFiles/fl_lib_audio.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[  5%] Built target fl_lib_set
[  8%] Performing update step for 'pybind11'
[ 37%] Built target fl_lib_sequence
[ 40%] No configure step for 'pybind11'
[ 43%] No build step for 'pybind11'
[ 45%] No install step for 'pybind11'
[ 48%] Completed 'pybind11'
[ 56%] Built target pybind11
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2
Traceback (most recent call last):
  File "setup.py", line 129, in <module>
    python_requires=">=3.6",
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 148, in setup
    return run_commands(dist)
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
    dist.run_commands()
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
    self.run_command(cmd)
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/dist.py", line 1214, in run_command
    super().run_command(command)
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
    cmd_obj.run()
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/home/ai-labs-/conda_envs/base/lib/python3.7/site-packages/setuptools/command/install.py", line 123, in do_egg_install
    self.run_command('bdist_egg')
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/dist.py", line 1214, in run_command
    super().run_command(command)
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
    cmd_obj.run()
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 165, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
    self.run_command(cmdname)
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/dist.py", line 1214, in run_command
    super().run_command(command)
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
    cmd_obj.run()
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/_distutils/command/install_lib.py", line 107, in build
    self.run_command('build_ext')
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/dist.py", line 1214, in run_command
    super().run_command(command)
  File "/home/ai-labs/conda_envs/base/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
    cmd_obj.run()
  File "setup.py", line 56, in run
    self.build_extensions(ext)
  File "setup.py", line 105, in build_extensions
    ["cmake", "--build", "."] + build_args, cwd=self.build_temp
  File "/home/ai-labs/conda_envs/base/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j4']' returned non-zero exit status 2.

I really, truly appreciate it if you can help me to finish this last part or hint something that I can fix it .

[benam2]

Any help here please @jacobkahn

[benam2]

Any update on this please??? This is really the worse experience ever :( how installing a package is that challenging!!

[jacobkahn]

@benam2 — the build message is pretty clear:

make[2]: *** No rule to make target '/home/ai-labs/fftw-3.3.10/build/libfftw3.so.3', needed by '../lib.linux-x86_64-3.7/libfl_lib_audio.so.0.3.2'. Stop.

This is a known issue with FFTW and isn't an issue with Flashlight -- see https://github.com/FFTW/fftw3/issues/130. You can try building FFTW from source or installing a version that ships FFTW3LibraryDepends.cmake which will properly define the library target make is failing to find a rule for.

If you're looking for beam search decoding and not audio featurization components, Python bindings for that have been moved to https://github.com/flashlight/text/.

[benam2]

@jacobkahn You are totally right. The problem is that for some reason building the FFTW3 raises error that's why I stopped using the GitHub to build.

What I finally did to install the flashlight with the help of your suggestion is that to install fftw3 with tools and then sudo cp FFTW3LibraryDepends.cmake /usr/local/lib/cmake/fftw3/

Now flashlight apparently got installed into my coda environment.

But my original problem has not been fixed yet. I have installed flashlight 1,5 month ago and faced with NameError: name 'CriterionType' is not defined for inferencing an audio for the transcription https://github.com/flashlight/flashlight/issues/1003

I started from scratch to install flashlight again however that error still persist. Do you have any insight what could be wrong?

I really appreciate if you can just spend a few minutes.

[jacobkahn]

@benam2 — are you trying to do decoding? CriterionType is defined in Flashlight Text -- installations instructions are here. Quickstart instructions for bindings here.

[benam2]

@jacobkahn I'm trying to use s3prl package to train ASR models. Do you think I should install Flashlight text instead?

[benam2]

@jacobkahn could you please share your idea with me? the reason I started installing flashlight python binding is that I've given the instruction from someone who had ran the code. Thats why I did not look into the alternative.

Now I think he may have just forgot to share this part with me. And the reason I don't install the flashlight text binging without confirmation is that I'm afraid just add a bunch of library that confuses and make it worse. Thats why I really appreciate it if you could help

[jacobkahn]

@benam2 — s3prl depends on Flashlight: Flashlight Text is a requirement, but it isn't clearly documented -- thanks for the callout. I'll send a PR updating that shortly.

[benam2]

@jacobkahn I do apologize for asking so many questions. So for s3prl I need to install the

1. build flashlight
2. install python bindings
3. And bindings for flashlight text

And since I have done the first two, I will need to do the flashlight text as well. Is this correct?

[jacobkahn]

@benam2 — after looking at s3prl, you don't need to build Flashlight or the Python bindings in this repo -- you only need Flashlight Text -- quickstart here.

[benam2]

I really don't know how to thank you :), and I hope you don't mind if I mention you in case I faced with error (hopefully I won't )

Thank again @jacobkahn

[benam2]

@jacobkahn I just ran these:

git clone https://github.com/flashlight/text && cd text
cd bindings/python
python3 setup.py install

And then after running the asr this error raises:

[Override] - config.downstream_expert.datarc.decoder_args.decoder_type = kenlm
[Override] - config.downstream_expert.datarc.decoder_args.kenlm_model = ./language_model/4-gram.arpa.gz
[Override] - config.downstream_expert.datarc.decoder_args.lexicon = ./language_model/librispeech_lexicon.lst
/home/ai-labs-sgoudarzvand/.conda/envs/asr/lib/python3.7/site-packages/transformers/configuration_utils.py:365: UserWarning: Passing `gradient_checkpointing` to a config initialization is deprecated and will be removed in v5 Transformers. Using `model.gradient_checkpointing_enable()` instead, or if you are using the `Trainer` API, pass `gradient_checkpointing=True` in your `TrainingArguments`.
  "Passing `gradient_checkpointing` to a config initialization is deprecated and will be removed in v5 "
Some weights of the model checkpoint at facebook/wav2vec2-base were not used when initializing Wav2Vec2Model: ['quantizer.codevectors', 'quantizer.weight_proj.bias', 'project_hid.weight', 'quantizer.weight_proj.weight', 'project_q.weight', 'project_q.bias', 'project_hid.bias']
- This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
[Featurizer for <class 's3prl.upstream.wav2vec2_hug.expert.UpstreamExpert'>] - The input upstream is only for initialization and not saved in this nn.Module
[Featurizer for <class 's3prl.upstream.wav2vec2_hug.expert.UpstreamExpert'>] - Take a list of 13 features and weighted sum them.
[Runner] - Loading Featurizer weights from the previous experiment
/home/ai-labs-sgoudarzvand/Desktop/ASR/s3prl/s3prl/downstream/asr/w2l_decoder.py:42: UserWarning: flashlight python bindings are required to use this functionality. Please install from https://github.com/facebookresearch/flashlight/tree/master/bindings/python
  "flashlight python bindings are required to use this functionality. Please install from [https://github.com/facebookresearch/flashlight/tree/master/bindings/python"](https://github.com/facebookresearch/flashlight/tree/master/bindings/python%22)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
/tmp/ipykernel_15148/2167564801.py in <module>
     31 
     32 upstream = 'wav2vec2_hug_base_960'
---> 33 runner = Runner(args, config)

~/Desktop/ASR/s3prl/s3prl/downstream/runner.py in __init__(self, args, config)
     50         self.upstream = self._get_upstream()
     51         self.featurizer = self._get_featurizer()
---> 52         self.downstream = self._get_downstream()
     53         self.all_entries = [self.upstream, self.featurizer, self.downstream]
     54 

~/Desktop/ASR/s3prl/s3prl/downstream/runner.py in _get_downstream(self)
    131             upstream_rate = self.featurizer.model.downsample_rate,
    132             **self.config,
--> 133             **vars(self.args)
    134         ).to(self.args.device)
    135 

~/Desktop/ASR/s3prl/s3prl/downstream/asr/expert.py in __init__(self, upstream_dim, upstream_rate, downstream_expert, expdir, **kwargs)
     97         )
     98         decoder_args = self.datarc.get('decoder_args')
---> 99         self.decoder = get_decoder(decoder_args, self.dictionary)
    100         self.register_buffer('best_score', torch.ones(1) * 100)
    101 

~/Desktop/ASR/s3prl/s3prl/downstream/asr/expert.py in get_decoder(decoder_args_dict, dictionary)
     34         if isinstance(decoder_args.unk_weight, str):
     35             decoder_args.unk_weight = eval(decoder_args.unk_weight)
---> 36         return W2lKenLMDecoder(decoder_args, dictionary)
     37 
     38     return None

~/Desktop/ASR/s3prl/s3prl/downstream/asr/w2l_decoder.py in __init__(self, args, tgt_dict)
    124 class W2lKenLMDecoder(W2lDecoder):
    125     def __init__(self, args, tgt_dict):
--> 126         super().__init__(args, tgt_dict)
    127 
    128         self.unit_lm = getattr(args, "unit_lm", False)

~/Desktop/ASR/s3prl/s3prl/downstream/asr/w2l_decoder.py in __init__(self, args, tgt_dict)
     53 
     54         # criterion-specific init
---> 55         self.criterion_type = CriterionType.CTC
     56         self.blank = (
     57             tgt_dict.index("<ctc_blank>")

NameError: name 'CriterionType' is not defined

I can't get from the error itself what could be missing @jacobkahn

[benam2]

Sorry again @jacobkahn but if you have an even a remote idea what else could be related to this error, I would really really appreciate if you could share with me. Its been too long I got stuck on this project .