transferwise / hisel

Feature selection tool based on Hilbert-Schmidt Independence Criterion
Apache License 2.0
2 stars 0 forks source link

Github tests are inconsistent with intended usage and they occasionally fail #14

Closed claudio-tw closed 1 year ago

claudio-tw commented 1 year ago

As part of github actions, the script run_tests.yaml installs the dependencies of hisel using pip and the requirement file requirements-test.txt. More precisely, here is a snippet from run_tests.yaml:

    steps:
      # GHA leaves mess behind. We need to clear those or our we end up having unexpected behaviours.
      # REMOVE ME when A&T uses ephemeral runners
      - name: Clear dirty runner
        # Clear everything, both home and working directory
        run: |
          rm -rfv ~/* ./* || true

      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Installing hisel dependencies
        run: |
          virtualenv hisel-env
          source hisel-env/bin/activate
          python3 -m pip install --upgrade pip wheel setuptools
          python3 -m pip install -r requirements-test.txt

This gives two problems:

  1. Dependency management in github actions is not aligned to the intended way to manage dependencies for hisel, which is via poetry.
  2. The tests run with Python 3.7, although there is no intention to support this version (The supported version is >=3.8, <3.12).

Moreover, since the version of numpy will be different from what specified in the pyproject.toml file, the selection algorithm that is tested as part of the github actions will not achieve the same level of accuracy obtained by the supported implementation with Python >=3.8, <3.12. This can cause the tests to fail (in particular select_test.py), and this might compromise the trust that people have in hisel.

How can we fix this issue? One way would be to use poetry in run_tests.yaml, but I do not know how to do that. In particular, we would need, say, Python 3.9 in the machine where the github actions run.

claudio-tw commented 1 year ago

This has been resolved in Use poetry in tests of github workflow #13