A few tests fails

[zapta]

When I run the command pytest apio test at the root of apio_dev I am getting a few errors. Is this the right way to run the tests? Are they actually broken or it's something on my side? (If broken, I can try fixing them).

===================================================================== short test summary info ======================================================================
FAILED test/code_commands/test_build.py::test_build_board - AssertionError: assert 'install oss-cad-suite' in 'Info: No apio.ini file\nInfo: No apio.ini file\nNo top module given\n\nOption 1: Pass it as a parameter\n   ...
FAILED test/code_commands/test_build.py::test_build_complete1 - AssertionError: assert 'install oss-cad-suite' in 'Info: No apio.ini file\nInfo: No apio.ini file\nNo top module given\n\nOption 1: Pass it as a parameter\n   ...
FAILED test/code_commands/test_time.py::test_time_board - AssertionError: assert 'apio install oss-cad-suite' in 'Info: No apio.ini file\nInfo: No apio.ini file\nNo top module given\n\nOption 1: Pass it as a parameter...
FAILED test/code_commands/test_verify.py::test_verify - AssertionError: assert 'apio install oss-cad-suite' in 'Info: No apio.ini file\nInfo: No apio.ini file\nNo top module given\n\nOption 1: Pass it as a parameter...
FAILED test/env_commands/test_config.py::test_config - assert 2 == 0
=================================================================== 5 failed, 27 passed in 6.67s ===================================================================

[zapta]

Here is for example a modified test_build_board() test. I am getting the error yosys: command not found instead of install oss-cad-suite. I guess it tries to run yosys before checking the existence of the package.

def test_build_board(clirunner, configenv):
    """Test: apio build --board icezum
    No oss-cad-suite package is installed
    """

    with clirunner.isolated_filesystem():

        # -- Config the environment (conftest.configenv())
        configenv()

        # Make sure wd don't have the oss cad suit installed.
        result = clirunner.invoke(cmd_uninstall, input='y')
        assert result.exit_code == 0, result.output

        # Create a dummy source file.
        print('''
              module main(input A, output B);
              assign B = !A;
              endmodule
              ''',  file=open("main.v", 'w'))

        # -- Execute "apio build --board icezum"
        result = clirunner.invoke(cmd_build, ["--board", "icezum", "--top-module", "main"])

        # -- Check the result
        assert result.exit_code != 0
        assert "install oss-cad-suite" in result.output

[zapta]

... seems to be related to this code. If running in native mode, the packages are not verified. My test runs in native mode. I don't know what it means.

        # -- Check the configuration mode
        if self.proj.exe_mode == "default":
            # Run on `default` config mode
            # -- Check if the necessary packages are installed
            if not util.resolve_packages(
                packages,
                self.profile.packages,
                self.resources.distribution.get("packages"),
            ):
                # Exit if a package is not installed
                raise AttributeError("Package not installed")
        else:
            click.secho("Info: native exe mode")

Test log

/Users/user/projects/apio_dev/repo $ pytest test/code_commands/test_build.py::test_build_board
======================================================================= test session starts ========================================================================
platform darwin -- Python 3.12.1, pytest-8.2.2, pluggy-1.5.0
rootdir: /Users/user/projects/apio_dev/repo
configfile: pyproject.toml
collected 1 item                                                                                                                                                   

test/code_commands/test_build.py F                                                                                                                           [100%]

============================================================================= FAILURES =============================================================================
_________________________________________________________________________ test_build_board _________________________________________________________________________

clirunner = <click.testing.CliRunner object at 0x10332fc80>, configenv = <function configenv.<locals>.decorator at 0x103460f40>

    def test_build_board(clirunner, configenv):
        """Test: apio build --board icezum
        No oss-cad-suite package is installed
        """

        with clirunner.isolated_filesystem():

            # -- Config the environment (conftest.configenv())
            configenv()

            # Make sure wd don't have the oss cad suit installed.
            result = clirunner.invoke(cmd_uninstall, input='y')
            assert result.exit_code == 0, result.output

            # Create a dummy source file.
            print('''
                  module main(input A, output B);
                  assign B = !A;
                  endmodule
                  ''',  file=open("main.v", 'w'))

            # -- Execute "apio build --board icezum"
            result = clirunner.invoke(cmd_build, ["--board", "icezum", "--top-module", "main"])

            print(f"{clirunner = }")
            print(f"{type(result) = }")

            print(f"{result.output = }")

            # -- Check the result
            assert result.exit_code != 0
>           assert "install oss-cad-suite" in result.output
E           AssertionError: assert 'install oss-cad-suite' in 'Info: No apio.ini file\nInfo: No apio.ini file\nInfo: native exe mode\n[Thu Jun 13 12:35:15 2024] Processing icezum\n...cons: *** [hardware.json] Error 127\n========================= [ ERROR ] Took 0.06 seconds =========================\n'
E            +  where 'Info: No apio.ini file\nInfo: No apio.ini file\nInfo: native exe mode\n[Thu Jun 13 12:35:15 2024] Processing icezum\n...cons: *** [hardware.json] Error 127\n========================= [ ERROR ] Took 0.06 seconds =========================\n' = <Result SystemExit(2)>.output

test/code_commands/test_build.py:86: AssertionError
----------------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------------

  --> configenv():
      apio working directory: /private/var/folders/6v/ck129h0s0g77x56llnrrfgwc0000gn/T/tmpyvpi00k5/ ñ
clirunner = <click.testing.CliRunner object at 0x10332fc80>
type(result) = <class 'click.testing.Result'>
result.output = 'Info: No apio.ini file\nInfo: No apio.ini file\nInfo: native exe mode\n[Thu Jun 13 12:35:15 2024] Processing icezum\n--------------------------------------------------------------------------------\n\n---> WARNING: no PCF file found (.pcf)\n\nyosys -p "synth_ice40 -top main -json hardware.json" -q main.v\nsh: yosys: command not found\nscons: *** [hardware.json] Error 127\n========================= [ ERROR ] Took 0.06 seconds =========================\n'
===================================================================== short test summary info ======================================================================
FAILED test/code_commands/test_build.py::test_build_board - AssertionError: assert 'install oss-cad-suite' in 'Info: No apio.ini file\nInfo: No apio.ini file\nInfo: native exe mode\n[Thu Jun 13 12:35:15 2024] Processing...
======================================================================== 1 failed in 0.12s =========================================================================
/Users/user/projects/apio_dev/repo $ 

[zapta]

... the tests don't use apio.ini, exe_mode is None, and the following code skips the package check.

if self.proj.exe_mode == "default":

Should it be this instead?

if self.proj.exe_mode != "native":

[zapta]

I sent this pull request that fixes the tests. It has to do with 'native' exe mode. The PR supports it as before but can we drop it for simplicity?

https://github.com/FPGAwars/apio/pull/383

[Obijuan]

I sent this pull request that fixes the tests. It has to do with 'native' exe mode. The PR supports it as before but can we drop it for simplicity?

Let's keep the native mode exe by the moment. In this mode the installed apio packages are ignored and the tools installed in the system are used instead (for example you can install oss-cad-suite natively in your os for testing a new version). It is an advance feature for developers. It is not supposed to be user by normal users

[Obijuan]

I will answer all your pending questions when I finish releasing Icestudio 0.12. Once it is done, we will start a new cycle and we can rethink apio/icestudio

[zapta]

Sounds good.

On Fri, Jun 14, 2024 at 2:01 AM Juan Gonzalez-Gomez < @.***> wrote:

I will answer all your pending questions when I finish releasing Icestudio 0.12. Once it is done, we will start a new cycle and we can rethink apio/icestudio

— Reply to this email directly, view it on GitHub https://github.com/FPGAwars/apio/issues/382#issuecomment-2167580481, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQVMQLUU3E37DSV2FG75BLZHKWN3AVCNFSM6AAAAABJIYDCN6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRXGU4DANBYGE . You are receiving this because you authored the thread.Message ID: @.***>

[zapta]

All tests pass now. Closing.

native mode will be discussed in a separate issue.