YAY!

YAY! is a high level parser combinator based PHP preprocessor that allows anyone to augment PHP with PHP :boom:

This means that language features could be distributed as composer packages (as long as the macro based implementations can be expressed in pure PHP code, and the implementation is fast enough).

Roadmap.

Set Up

composer require yay/yay:dev-master

Usage

Command Line

yay some/file/with/macros.php >> target/file.php

Runtime Mode

The "runtime" mode is W.I.P and will use stream wrappers along with composer integration in order to preprocess every file that gets included. It may have some opcache/cache support, so files will be only preprocessed/expanded once and when needed.

See feature progress at issue #11.

How it works

Very Simple Example

Every macro consist of a matcher and an expander that when executed allows you to augment PHP. Consider the simplest example possible:

$(macro :unsafe) { $ } >> { $this } // this shorthand

The macro is basically expanding a literal $ token to $this. The following code would expand to:

// source                                |   // expansion
class Foo {                              |   class Foo {
    protected $a = 1, $b = 2, $c = 3;    |       protected $a = 1, $b = 2, $c = 3;
                                         |        
    function getProduct(): int {         |       function getProduct(): int {
        return $->a * $->b * $->c;       |           return $this->a * $this->b *$this->c;
    }                                    |       }
}                                        |   }

Notice that the :unsafe tag is necessary to avoid macro hygiene on $this expansion.

This macro is actually very naive, a more producion ready version would be:

$(macro :unsafe){
    $ // litterally matches '$'
    // but not followed by:
    $(not(token(T_VARIABLE))) // avoids var var false positives such as '$$foo'
    $(not(token('{'))) // avoids false positives such as '${foo}'
} >> {
    $this
}

Simple Example

Apart from literal characher sequences, it's also possible to match specific token types using the token matcher in the form of $(TOKEN_TYPE as label).

The following macro matches token sequences like __swap($x, $y) or __swap($foo, $bar):

$(macro) {
    __swap ( $(T_VARIABLE as A) , $(T_VARIABLE as B) )
} >> {
    (list($(A), $(B)) = [$(B), $(A)])
}

The expansion should be pretty obvious:

// source              |    // expansion
__swap($foo, $bar);    |    (list($foo, $bar) = [$bar, $foo]);

Another Simple Example

To implement unless we need to match the literal unless keyword followed by a layer of tokens between parentheses (...) and a block of code {...}. Fortunately, the macro DSL has a very straightforward layer matching construct:

$(macro) {
    unless ($(layer() as expression)) { $(layer() as body) }
} >> {
    if (! ($(expression))) {
        $(body)
    }
}

The macro in action:

// source                   |   // expansion
unless ($x === 1) {         |   if (! ($x === 1)) {
    echo "\$x is not 1";    |       echo "\$x is not 1";
}                           |   }

PS: Please don't implement "unless". This is here just for didactic reasons.

Advanced Example

A more complex example could be porting enums from the future to PHP with a syntax like:

enum Fruits {
    Apple,
    Orange
}

var_dump(\Fruits::Orange <=> \Fruits::Apple);

So, syntactically, enums are declared with the literal enum word followed by a T_STRING and a comma separated list of identifiers withing braces such as {A, B, C}.

YAY uses parser combinators internally for everything and these more high level parsers are fully exposed on macro declarations. Our enum macro will need high level matchers like ls() and label() combined to match the desired syntax, like so:

$(macro) {
    enum $(T_STRING as name) {
        $(
            // ls() matches a delimited list
            // in this case a list of label() delimited by ',' such as `foo, bar, baz`
            ls
            (
                label() as field
                ,
                token(',')
            )
            as fields
        )
    }
} >> {
    "it works";
}

The macro is already capable to match the enum syntax:

// source                      // expansion
enum Order {ASC, DESC};    |   "it works";

I won't explain how enums are implemented, you can read the RFC if you wish and then see how the expansion below works:

// things here would normally be under a namespace, but since we want a concise example...

interface Enum
{
}

function enum_field_or_class_constant(string $class, string $field)
{
    return (\in_array(\Enum::class, \class_implements($class)) ? $class::$field() : \constant("{$class}::{$field}"));
}

$(macro :unsafe) {
    // the enum declaration
    enum $(T_STRING as name) {
        $(
            ls
            (
                label() as field
                ,
                token(',')
            )
            as fields
        )
    }
} >> {
    class $(name) implements Enum {
        private static $registry;

        private function __construct() {}

        static function __callStatic(string $type, array $args) : self {
            if(! self::$registry) {
                self::$registry = new \stdclass;
                $(fields ... {
                    self::$registry->$(field) = new class extends $(name) {};
                })
            }

            if (isset(self::$registry->$type)) return self::$registry->$type;

            throw new \Exception(sprintf('Undefined enum type %s->%s', __CLASS__, $type));
        }
    }
}

$(macro) {
    $(
        // sequence that matches the enum field access syntax:
        chain(
            ns() as class, // matches a namespace
            token(T_DOUBLE_COLON), // matches T_DOUBLE_COLON used for static access
            not(class), // avoids matching `Foo::class`, class resolution syntax
            label() as field, // matches the enum field name
            not(token('(')) // avoids matching static method calls such as `Foo::bar()`
        )
    )
} >> {
    \enum_field_or_class_constant($(class)::class, $$(stringify($(field))))
}

More examples within the phpt tests folder https://github.com/marcioAlmada/yay/tree/master/tests/phpt

FAQ

Why "YAY!"?

- PHP with feature "x": yay or nay? :wink:

Where is the documentation?

A cookbook is on the making

Why are you working on this?

Because it's being fun. It may become useful. Because we can™.

Conclusion

For now this is an experiment about how to build a high level preprocessor DSL using parser combinators on a languages like PHP. Why?

PHP is very far from being homoiconic and therefore requires complex deterministic parsing and a big AST implementation with a node visitor API to modify source code - and in the end, you're not even able to easily process unknown syntax ¯\_(⊙_ʖ⊙)_/¯.

That's why this project was born. It was also part of the challenge:

Create a minimalistic architecture that exposes a subset of the internal components, that power the preprocessor itself, to the user DSL.
Create parser combinators with decent error reporting and grammar invalidation, because of 1

marcioAlmada / yay

readme