Support fully general boolean filters in Gallio FilterParser.

[GoogleCodeExporter]

This feature will be in MbUnit v3.  We will be releasing an Alpha version
of MbUnit v3 within the next couple of weeks.
The nice thing is that MbUnit v3 (actually the Gallio Test Automation
Platform underneath it) can run MbUnit v2 tests unmodified.

The syntax for OR currently looks a bit like this:

Gallio.Echo.exe SomeAssembly.dll /f:Category=Foo,Bar,Blah

You can AND filters together like this:

Gallio.Echo.exe SomeAssembly.dll
/f:Category=Foo,Bar;Type=SomeType,SomeOtherType

Assuming a != condition is added, you examples could get translated to
something like this:

WinXP or WinVista: "Category=WinXP, WinVista"
NOT Excel: "Category!=Excel2003, Excel2007"
Debug or Release: "Category=Debug, Release"
WinXP and Debug and not Excel: "Category=WinXP, WinVista; Category=Debug;
Category!=Excel2003"

To make things fully general we'll need grouping operators and combinators.
 This is already fully supported by the infrastructure but not available in
the syntax:

(NOT (WinXP AND Excel2003)) OR Foo: "! (Category=WinXP; Category=Excel2003)
| Category=Foo"

Jeff.

--------------------------------------------------------------------------------
From: MbUnitUser@googlegroups.com [mailto:MbUnitUser@googlegroups.com] On
Behalf Of Leonid Lastovkin
Sent: Monday, October 29, 2007 5:22 PM
To: MbUnitUser@googlegroups.com
Subject: MbUnit Is it possible to "AND" categories (in addition to
"OR"-ing) for /fc and /ec flags of the MbUnit.Cons/GUI.exe ?

I need to partition (in proper mathematical sense - mutually exclusive and
collectively exhaustive)
the tests (actually test fixtures) that I have while not separating them
into different assemblies.
Each set (or a pair of sets in the case of Debug vs Release) of test
fixtures will run on a designated test host 
(with different versions of Windows and Excel installed).
Each will spit out its own xml result, which will get merged into a single
report.
Each test host will have to work with the exact same C# project containing
tests. 
Each host will be able to compile the project in Debug | Release or other, 
but will have to figure out how to run only the tests that are appropriate
for the versions of Windows and Office installed.

Right now my 8 categories are: 

1) Tests that are meant to run under Windows XP, with Office 2003 in Debug
mode.
...
8) Tests that are meant to run under Windows Vista, with Office 2007 in
Release mode.

I can add several categories to each test fixture; for example: 

[TestFixture]
[FixtureCategory("WinXP")]
[FixtureCategory("Excel2003")]
[FixtureCategory("Debug")]

I may want to complicate things slightly by introducing other categories: 
  *    "AnyWin" = "WinXP" OR "WinVista", 
 **    "NoExcel" =  NOT ("Excel2003" OR "Excel 2007"), 
***    "AnyTarget" = "Debug" OR "Release", 

I can use the switches /fc = /filter-category and /ec = /exclude-category
to achieve some filtering, but I cannot do everything that I would like to do.
If I understand the documentation correctly, multiple categories get OR-ed
when I supply a comma-separated list of them. 

For example, the following switch for the MbUnit.Cons.exe: 
    "/fc:x1,x2,x3,...xn /ec:y1,y2,y3,...ym" is logically equivalent to:

[(x1 OR x2 OR x3 ... OR xn)] AND [( NOT (y1 OR y2 OR y3 ... OR ym))] 

The structure of this logical expression is restricted to at most: a single
AND, a single NOT, two (though arbitrary long) ORs and 10 parenthesis's. 
I cannot create a category filter like: (WinXP only) and (Debug only) and
no Excel. 

================================================================================
=========================

Or can I? Am I missing something about the usage of MbUnit.Cons/GUI.exe ?
I would find this feature useful. 

Thanks.

Original issue reported on code.google.com by jeff.br...@gmail.com on 30 Oct 2007 at 2:04

[GoogleCodeExporter]

Original comment by jeff.br...@gmail.com on 30 Oct 2007 at 2:13

• Added labels: Component-Gallio
• Removed labels: Component-Core

[GoogleCodeExporter]

Julian, can we get the != syntax in for Alpha 1?  The rest can wait.

Original comment by jeff.br...@gmail.com on 30 Oct 2007 at 9:22

[GoogleCodeExporter]

After discussing this with Julian, it seems the best approach is to just 
rewrite the
parser code and to use a more composable syntax.  Here's a quick and dirty
suggestion.  Looks like we can get this into Alpha 1 too.  Nice.  ;-)

This syntax looks LL(1) to me, so we should be able to implement a simple 
recursive
descent parser in a couple dozen lines that covers all of these cases.

<word>         ::= [a-zA-Z0-9_\-]+  # unquoted word, this regex probably isn't 
good
enough yet...

<string>       ::= <word>
                 | '"' ([^\"]|\\.)* '"'
                 | "'" ([^\']|\\.)* "'"

<key>          ::= <string>

<value>        ::= <string>     # value specified by exact string
                 | "~" <string> # value specified by regex

<match>        ::= <value>      # inclusion match of a value
                 | "!" <value>  # exclusion match of a value

<matchSeq>     ::= <match>
                 | <match> "," <matchSeq>

<filter>       ::= <key> ":" <matchSeq>
                 | <filter> "|" <filter>  # combine filters with OR
                 | <filter> "&" <filter>  # combine filters with AND
                 | "!" <filter>           # negate filter
                 | "(" <filter> ")"       # grouping

Where NOT has highest precedence, then AND, then OR and these operators are
left-associative.

Original comment by jeff.br...@gmail.com on 31 Oct 2007 at 2:09

• Added labels: Milestone-3.0-a1
• Removed labels: Milestone-3.0-a2

[GoogleCodeExporter]

Nix value-level exclusion.  It's too confusing.  Filter-level negation is fine.

Original comment by jeff.br...@gmail.com on 9 Nov 2007 at 10:53

[GoogleCodeExporter]

Note: Should allow "and", "or", and "not" to be used as synonyms for "&", "|" 
and "!"
to simplify embedding of complex filter specifications within XML.

Original comment by jeff.br...@gmail.com on 12 Nov 2007 at 8:11

[GoogleCodeExporter]

I implemented this feature and updated the related documentation.

Original comment by julian.h...@gmail.com on 26 Nov 2007 at 9:21

[GoogleCodeExporter]

Original comment by julian.h...@gmail.com on 26 Nov 2007 at 9:22

• Changed state: Fixed

[GoogleCodeExporter]

I added more unit tests last week and I'm working on improving the error 
reporting.

Original comment by julian.h...@gmail.com on 2 Dec 2007 at 10:54

[GoogleCodeExporter]

After talking with Jeff, we agreed on performing the following changes:
- Use /.../ instead of ~"..." for defining regular expressions, and /.../i for
case-insensitive matches
- Redefine the word characters to be any printable character that's not 
explicitly
reserved for other uses
- Introduce an ERROR token for lexer errors

Original comment by julian.h...@gmail.com on 3 Dec 2007 at 11:12

[GoogleCodeExporter]

Jeff suggested to simply throw out the "&", "|" and "!" operators since "and", 
"or"
and "not" are good enough.

Original comment by julian.h...@gmail.com on 4 Dec 2007 at 11:35

[GoogleCodeExporter]

Hehe... keeping in mind that my suggestion may or may not be valid.  I think I 
can
make a good case for it though.

;-)

Original comment by jeff.br...@gmail.com on 4 Dec 2007 at 11:36

• Changed state: Started

[GoogleCodeExporter]

Reopened the issue to address minor changes in the specs as described above.

Original comment by jeff.br...@gmail.com on 4 Dec 2007 at 11:37

• Added labels: Milestone-3.0-a2
• Removed labels: Milestone-3.0-a1

[GoogleCodeExporter]

6. Fix the following bug:

TestCase 'FilterFormatterTest.RoundTripFormatting(not (Abc: 123 and not (Def: 
456 or
not Ghi: 789))' failed: 
    System.Exception: Right bracket expected
    C:\Source\MbUnit\v3\src\Gallio\Gallio\Model\Filters\FilterParser.cs(223,0): at
Gallio.Model.Filters.FilterParser`1.MatchRightBracket(FilterLexer lexer)

The problem happens because the grammar's precendence rules are not quite 
correct.
I think they should be:

Filter -> OrFilter

OrFilter -> AndFilter 'or' OrFilter
         -> AndFilter

AndFilter -> ParenFilter 'or' AndFilter
          -> ParenFilter

NotFilter -> 'not' NotFilter
          -> ParenFilter

ParenFilter -> '(' OrFilter ')'
            -> Filter

SimpleFilter -> Key ':' Value

Here are some more test cases for you to work with:

- "not (Abc: 123 and not (Def: 456 or not Ghi: 789)"
- "not (not not (((Abc: def))))"
- "Abc: /123 456 \/ 789/i"
- "* and * or * and *"  ===  "(* and *) or (* and *)"
- "* or * and * or *"  ===  "* or (* and *) or *"
- "not * or not * and *"  ===  "(not *) or ((not *) and *)"

When you're done making these changes, you'll want to run the 
FilterFormatterTests to
check.  I wrote them under the assumption that the parser would change as 
specified.

Jeff. 

P.S.  The new "*" token means AnyFilter.  It's purpose is to provide a non-empty
token that represents a totally inclusive filter.  With the "*" token in place, 
we
can then disallow empty filter specifications altogether.  So "*" on its own 
becomes
the default filter.  This is nice because it ensures the filter is always 
printable.

-----Original Message-----
From: Jeff Brown
Sent: Thursday, December 27, 2007 2:30 PM
To: 'Julián Hidalgo'
Subject: RE: Filter Parser work.

3. Remove support for &, | and !.

4. Update the documentation accordingly.

5. Add a test for "ExactType" which is like "Type" but does not include derived 
types.

Jeff. 

-----Original Message-----
From: Jeff Brown
Sent: Thursday, December 27, 2007 2:22 PM
To: 'Julián Hidalgo'
Subject: Filter Parser work.

A couple of remaining items:

1. It doesn't look like the new Regex syntax is supported yet.  eg. /some 
regex/i  
(no quotes needed)

2. It also seems we'll need a token for an "any" filter.  So let's make "*" 
produce
an AnyFilter<T>.

Sound good?  I'd like to get this stuff finished in time for Alpha 2 this 
weekend.

Jeff.

Original comment by jeff.br...@gmail.com on 6 Jan 2008 at 7:22

[GoogleCodeExporter]

> 6. Fix the following bug:
> 
> TestCase 'FilterFormatterTest.RoundTripFormatting(not (Abc: 123 and not (Def: 
456
or not Ghi: 789))' failed: 
>   System.Exception: Right bracket expected
>   C:\Source\MbUnit\v3\src\Gallio\Gallio\Model\Filters\FilterParser.cs(223,0): at
Gallio.Model.Filters.FilterParser`1.MatchRightBracket(FilterLexer lexer)

Actually there was a missing bracket:
not (Abc: 123 and not (Def: 456 or not Ghi: 789)
Should have been
not (Abc: 123 and not (Def: 456 or not Ghi: 789))

:)

Original comment by julian.h...@gmail.com on 14 Jan 2008 at 2:37

[GoogleCodeExporter]

I implemented all the changes.

Original comment by julian.h...@gmail.com on 14 Jan 2008 at 7:08

[GoogleCodeExporter]

Original comment by jeff.br...@gmail.com on 22 Feb 2008 at 8:05

• Changed state: Fixed