Parse tree selectors: blacklist vs whitelist

[drtconway]

Hi.

First, thanks for an awesome library. I've been massively impressed.

I have a really big grammar with about 1200 productions. As I go, I'm refining the grammar to produce a more convenient parse tree. I fairly quickly concluded that it would be more convenient to blacklist productions for parse tree nodes, rather than whitelist them.

From the documentation, a whitelist can me made in the following way:

    template< typename Rule > struct whitelist : std::false_type {};
    template<> struct whitelist<identifier>  : std::true_type {};
    template<> struct whitelist<number>  : std::true_type {};
    ...

So I concluded that it should be straight-forward to invert things to make a blacklist:

    template< typename Rule > struct blacklist : std::true_type {};
    template<> struct blacklist<sep>  : std::false_type {};
    template<> struct blacklist<alpha>  : std::false_type {};
    template<> struct blacklist<digit>  : std::false_type {};
    ...

where the list of explicit template instantiations should be the complement of those in the whitelist.

I have attached an example program with a simple expression grammar, with a little traversal to convert to a simple json-like form. The program parses the input twice - once with a whitelist, and once with a blacklist. Naively, I would expect the output of the two to be the same, but clearly they are not.

It makes sense what is happening - all the intermediate combinator forms are not marked in the blacklist, so they are included.

Is there a straight-forward way to modify my blacklist approach? If so, it'd be pretty useful.

Thanks, Tom.

blacklist.txt

[drtconway]

Of course, the act of creating and writing the test case, and describing the problem got me thinking.

Adding the following rules to the blacklist achieved the desired outcome:

    template<> struct blacklist<parenthesized>  : std::false_type {};
    template <typename... Rules>
    struct blacklist<tao::pegtl::seq<Rules...>> : std::false_type {};
    template <typename... Rules>
    struct blacklist<tao::pegtl::sor<Rules...>> : std::false_type {};
    template <typename... Rules>
    struct blacklist<tao::pegtl::opt<Rules...>> : std::false_type {};
    template <typename... Rules>
    struct blacklist<tao::pegtl::star<Rules...>> : std::false_type {};
    template <typename... Rules>
    struct blacklist<tao::pegtl::plus<Rules...>> : std::false_type {};
    template <typename... Rules>
    struct blacklist<detail::interleaved<Rules...>> : std::false_type {};

    template<> struct blacklist<tao::pegtl::ascii::alpha>  : std::false_type {};
    template<> struct blacklist<tao::pegtl::ascii::digit>  : std::false_type {};
    template<> struct blacklist<tao::pegtl::ascii::space>  : std::false_type {};

I'll leave this here in case it helps anyone else.

I guess a possible enhancement to the library might be to provide a template with all the library productions.

Tom.

[d-frey]

We don't have any direct support for what you are looking for. We already filter out internal types by default, but the user-visible rules should, by design, be indistinguishable from user-written rules. A true solution for the general case would require reflection so you could simple replace the default's std::true_type with a test for the class' namespace and only include those rules that are in your namespace. But since C++ still does not have proper reflection... sigh. Sorry.

[drtconway]

Cheers!