csukuangfj / kaldifeat

Kaldi-compatible online & offline feature extraction with PyTorch, supporting CUDA, batch processing, chunk processing, and autograd - Provide C++ & Python API
https://csukuangfj.github.io/kaldifeat
Other
186 stars 35 forks source link

`pip install kaldifeat`: CUDA not found error in GitHub CI #11

Closed pzelasko closed 2 years ago

pzelasko commented 2 years ago

I'm adding kaldifeat support in Lhotse in this PR: https://github.com/lhotse-speech/lhotse/pull/424

Please see the error log here:

https://github.com/lhotse-speech/lhotse/pull/424/checks?check_run_id=3930520740

csukuangfj commented 2 years ago

They are two approaches to fix this issue:

(1) Install a CPU version of PyTorch. By default, pip install torch installs a CUDA version. When we use CMake to find torch, it requires CUDA support; that's why it throws the above error.

Please see the code in icefall for how to install a CPU version of PyTorch. https://github.com/k2-fsa/icefall/blob/3effcb42253c72c9bcf7f53484ce6c98807555f7/.github/workflows/run-pretrained.yml#L52

You can first uninstall PyTorch in GitHub CI and reinstall it so that you don't need to change setup.py in lhotse. That is, put

pip uninstall torch
pip install torch==1.9.1+cpu -f https://download.pytorch.org/whl/torch_stable.html

after https://github.com/lhotse-speech/lhotse/blob/master/.github/workflows/unit_tests.yml#L35

(Note: You can make the version of PyTorch configurable, like what icefall is doing).

(2) This approach is more complicated. You need to install CUDA toolkit in GitHub CI. The good news is that the code already exists in k2.

Please refer to https://github.com/k2-fsa/k2/blob/master/.github/workflows/build.yml#L98 for how to install CUDA toolkit in GitHub CI.

      - name: Install CUDA Toolkit ${{ matrix.cuda }}
        env:
          cuda: ${{ matrix.cuda }}
        run: |
          source ./scripts/github_actions/install_cuda.sh
          echo "CUDA_HOME=${CUDA_HOME}" >> $GITHUB_ENV
          echo "${CUDA_HOME}/bin" >> $GITHUB_PATH
          echo "LD_LIBRARY_PATH=${CUDA_HOME}/lib:${CUDA_HOME}/lib64:${LD_LIBRARY_PATH}" >> $GITHUB_ENV
        shell: bash

I recommend the first approach. It is simpler.

pzelasko commented 2 years ago

Thanks a lot! I’ll check out the first one.

csukuangfj commented 2 years ago

Closing as it has been fixed.