jorgebucaran / fishtape

100% pure-Fish test runner
MIT License
347 stars 20 forks source link

Conditional test not running #42

Closed edouard-lopez closed 5 years ago

edouard-lopez commented 5 years ago

In pure we have some test cases conditioned by the fish version running. However, while migrating to 2.1.0 they stop working.

For instance, I got no output for the following

if fish_version_below '3.0.0'
    @test "_pure_prompt_ssh: displays 'user@hostname' when on SSH connection" (
       …
    ) = 0
end

However, if I add an echo whatever in the if block then I got some output

ok 2 _pure_prompt_ssh: displays 'user@hostname' when on SSH connection

if fish_version_below '3.0.0'
    echo whatever
    @test "_pure_prompt_ssh: displays 'user@hostname' when on SSH connection" (
       …
    ) = 0
end
jorgebucaran commented 5 years ago

@edouard-lopez You need to silence fish_version_below.

if fish_version_below '3.0.0' >/dev/null
    @test "_pure_prompt_ssh: displays 'user@hostname' when on SSH connection" (
       …
    ) = 0
end

The problem is fish_version_below doesn't add newlines to its output, distorting test results, which Fishtape is unable to parse.

fish_version_below 4.0.0
only fish <4.0.0: ⏎

As a rule of thumb, don't write to stdout inside your test files unless you are going to capture that output in a variable or do it inside a command substitution.

edouard-lopez commented 5 years ago

I updated the code to print to stderr, that fixes it.

However, I get messed-up outputs which I reckon are due to the tests being run as background jobs. So the output is no more prepend to the test result.

Is there a way to sync the fish_version_below with its test output?

jorgebucaran commented 5 years ago

@mesg is your friend. It's described (briefly) in the Usage section in the docs.

My fish_version_below would only set the exit status code to 0 or 1 without writing to stdout or stderr.

Then I'd do something like this:

if fish_version_below 3.0.0
    echo "fish $version"
    @test "_pure_prompt_ssh: displays 'user@hostname' when on SSH connection" (...) = 0
end