k2-fsa / kaldifst

Python wrapper for OpenFST and its extensions from Kaldi. Also support reading/writing ark/scp files
https://k2-fsa.github.io/kaldifst/
Other
47 stars 9 forks source link

The issue of pytest conflicting with kaldifst #61

Open frznkshn opened 4 months ago

frznkshn commented 4 months ago

Hi,

When I import pytest with kaldifst in a code repository and execute the command python3 run_test.py, it seems like the program gets stuck in a loop and doesn't return any result, nor does it give any errors. I would appreciate your opinion on what to do next. It appears that pytest has an issue with kaldifst—and generally with C++ Python wrappers.

# run_test.py

import kaldifst
import pytest

# Run pytest with coverage and generate reports
pytest.main(['--cov', '--cov-report', 'term', '--cov-report', 'xml:coverage.xml'])

What is shown in the terminal

===================================================================== test session starts =====================================================================
platform linux -- Python 3.8.16, pytest-7.4.4, pluggy-1.3.0
rootdir: /home
plugins: cov-4.1.0, hypothesis-6.92.2, hydra-core-1.1.2
collecting 1190 items
csukuangfj commented 4 months ago

it seems like the program gets stuck in a loop and doesn't return any result,

What is the expected result if you remove import kaldifst?

Does it work if you replace

import kaldifst

with

import numpy

or with

import torch

?

frznkshn commented 4 months ago

it seems like the program gets stuck in a loop and doesn't return any result,

What is the expected result if you remove import kaldifst?

Does it work if you replace

import kaldifst

with

import numpy

or with

import torch

?

Yes - When we replace kaldifst with numPy or torch or ..., the problem is resolved (a result is returned as the output of the test). Note that the program I'm trying to test coverage on includes several modules. I realized that pytest has issues running it, and after conducting investigations, I found that the problem lies with this Python wrapper. Then, I created a simple script as described above and found that even in this simple scenario, the program doesn't execute.

When we replace it with NumPy, we get such an output:

===================================================================== test session starts =====================================================================
platform linux -- Python 3.8.16, pytest-7.4.4, pluggy-1.3.0
rootdir: /home
plugins: cov-4.1.0, hypothesis-6.92.2, hydra-core-1.1.2
collected 1226 items / 3 errors
.
.
.
=================================================================== short test summary info ===================================================================
ERROR num2words/tests/test_cli.py
ERROR pyannote-audio-3.1/tests/test_run_notebooks.py
ERROR pyannote-audio-3.1/tests/tasks/test_reproducibility.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 3 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=============================================================== 4 warnings, 3 errors in 21.63s ================================================================