bats-core / bats-core

Bash Automated Testing System
https://bats-core.readthedocs.io/
Other
4.98k stars 424 forks source link

Tag filter does not allow to filter by top level tag #966

Open TLINDEN opened 3 months ago

TLINDEN commented 3 months ago

Describe the bug According to the documentation:

The colon is intended as a separator for (recursive) namespacing.

I added a couple of tags like simple:a, simple:b etc. Then I want to run all tests under the simple "tag namespace" by:

bats/bin/bats --filter-tags simple --verbose-run tom.bats

Which does not execute any test under simple: tags.

To Reproduce Steps to reproduce the behavior:

Test file:

#!/bin/bash

# bats test_tags=simple:a
@test "simple a" {
    run ls
    [ "$status" -eq 0 ]
}

# bats test_tags=simple:b
@test "simple b" {
    run ls
    [ "$status" -eq 0 ]
}

# bats test_tags=complex:a
@test "complex a" {
    run ls
    [ "$status" -eq 0 ]
}

# bats test_tags=complex:a
@test "complex a" {
    run ls
    [ "$status" -eq 0 ]
}

Execute:

bats/bin/bats --filter-tags simple --verbose-run tom.bats 

0 tests, 0 failures

However, specifying the whole "namespace" works:

bats/bin/bats --filter-tags simple:a --verbose-run tom.bats 
tom.bats
 ✓ simple a

1 test, 0 failures

Expected behavior

Bats should execute all tests tagged simple

Environment (please complete the following information):

TLINDEN commented 3 months ago

PS: I can work around the problem by using

# bats test_tags=simple,a
@test "simple a" {
    run ls
    [ "$status" -eq 0 ]
}

# bats test_tags=simple,b
@test "simple b" {
    run ls
    [ "$status" -eq 0 ]
}

That way I can now execute all tests tagged simple. However, it's not a hierarchy anymore. If any other test also has a b tag, then I am unable to address the specific test:

# bats test_tags=complex,a
@test "complex a" {
    run ls
    [ "$status" -eq 0 ]
}

# bats test_tags=complex,a
@test "complex a" {
    run ls
    [ "$status" -eq 0 ]
}

Now specifying --filter-tags a matches the "simple a" and "complex a" test:

bats/bin/bats --filter-tags a --verbose-run tom.bats 
tom.bats
 ✓ simple a
 ✓ complex a

2 tests, 0 failures

However, I can use the filter simple,a to just execute one test:

bats/bin/bats --filter-tags simple,a --verbose-run tom.bats 
tom.bats
 ✓ simple a

1 test, 0 failures

Maybe this is the intended behavior, but then the documentation is not clear enough about it.

See also https://github.com/bats-core/bats-core/issues/842

martin-schulze-vireso commented 2 months ago

I will mark this as a feature request. The separators were mostly intended as reading aid.