avocado-framework / avocado

Avocado is a set of tools and libraries to help with automated testing. One can call it a test framework with benefits. Native tests are written in Python and they follow the unittest pattern, but any executable can serve as a test.
https://avocado-framework.github.io/
Other
345 stars 343 forks source link

exec-test: allow setting of the working directory #5937

Open clebergnu opened 6 months ago

clebergnu commented 6 months ago

Is your feature request related to a problem? Please describe. The exec-test test type allows users to run an executable given by absolute path, or more commonly by relative path. In both cases, the current working directory inherited by Avocado or the specific test runner (say avocado-runner-exec-test) may not be appropriate to the executable.

One common situation is when using an executable that will depend on files located somewhere else in the filesystem. For instance, having a runnable's uri of make, together with a make target as an argument given to args, will depend on the location of the Makefile containing that target.

The goal here is to implement a way to control how exec-tests can set their current working directory, so that use cases such as make <target> can be defined with uri and args in a way that makes the execution relocatable (that is, does not depend on running from a specific location).

Describe the solution you'd like A configuration option such as runner.exectest.cwd can take a number of options that will control the current working for the executables themselves. For instance:

Describe alternatives you've considered None.

Additional information None

clebergnu commented 2 months ago

Hi @dgibson, can you share your opinion on this? Thanks!

dgibson commented 2 months ago

Hi @clebergnu, sorry it's taken me a while to look at this.

I don't think setting the cwd relative to the uri or one of the arguments is very useful.

Instead, what I think would be useful is to be able to set the cwd as a path relative to the directory if the .json itself. That way a .json can refer to things within (say) its own repository in a way that won't break if the whole repository is moved around. For simplicity/brevity, I'd prefer uri was also interpreted relative to the configured cwd - that's more or less necessarily true of any arguments.

For example, in some source tree test/tests.json might have:

[
    {
        "kind": "exec-test",
        "identifier": "build",
        "cwd": "..",
        "uri": "make",
        "args": ["all"]
    },
    {
        "kind": "exec-test",
        "identifier": "lint",
        "cwd": "..",
        "uri": "scripts/lintscript",
        "args": ["-C", "test/lintconfig"]
    }
]

I don't personally see a use for setting cwd based on kwargs, but it might be useful to someone, so I have no strong opinion on that option.

What might be useful, would be the ability to pass the test an environment variable (or Python kwarg, I guess) that's given as a path relative to the .json's location. Once inside the test the variable would have the absolute path, or a path relative to the actual cwd. This is technically unrelated to setting of the cwd, but overlaps in use case, since it could pass information that the test itself could use to control its cwd. e.g.

[
    {
        "kind": "exec-test",
        "identifier": "complicated-test",
        "uri": "./complicated_test",
        "env": { "REPOBASE": "{jsonpath}/.." }
    }
]

(the specific syntax I'm using there is just an example to get the idea across, I don't think it would be a good choice for a final syntax).