Proposal: testing @compileError

[PavelVozenilek]

Minor feature to test edge cases.

Zig allows custom made compiler errors. Example is here: http://andrewkelley.me/post/zig-programming-language-blurs-line-compile-time-run-time.html

There should be some easy way to check if they are implemented correctly.

---

My proposal:

test "expected to fail"
{
  something_generating_error();  /// DOES NOT COMPILE
}

If compiler reaches @compileError inside a test it should then look for comment DOES NOT COMPILE at the offending line. If it finds it there, all is OK, testing continues. If it doesn't, original error is reported, it is a bug to be fixed.

If the test compiles OK but contains DOES NOT COMPILE comment, then it should fail.

---

Alternative is special keyword for failing test, but this feels as overkill.

Feature should be limited to user written @compileError only: syntactic errors (like unbalanced parenthesis) should be always error, wrong use of the language (x = 1 + "abc") as well.

DOES NOT COMPILE comment outside a test means nothing.

Expected failed test would generate no code.

Failing tests could be certainly implemented as separately compiled files, but this is nuisance most people would avoid.

[thejoshwolfe]

@compileError() takes a msg parameter, so testing for such an error should also take a msg that must match. How about:

test "fork" {
    switch(builtin.os) {
        Os.linux, Os.darwin, Os.macosx, Os.ios => testFork(),
        else => @expextCompileError("Unsupported OS", testFork()),
    };
}

[PavelVozenilek]

One small improvement: compiler should also remove the line annotated as failing, and then try to compile (but not run) the remaining code. This would eliminate mistakes that would otherwise go unnoticed.

So this:

test "expected to fail"
{
  something_generating_error();  /// DOES NOT COMPILE
}

would be internally transformed into two tests:

test "expected to fail"
{
  xyz
  something_generating_error();  /// DOES NOT COMPILE
}

test "expected to compile, but not executed"
{
  xyz
}

[marler8997]

Not knowing this proposal existed, I had made a duplicate of this proposal here: https://github.com/ziglang/zig/issues/3144

Along with testing compile errors, it's important to note that this feature would also enable much more powerful generic specialization. With this builtin function, generics can now test types for any feature that does or does not compile. I've included some contrived examples to demonstrate some of the capabilities this enables.

Note that I'm not trying to say whether these examples are good or bad or even whether supporting them is good or bad. I'm just pointing out what this new builtin function would enable. I think testing for compile failures is probably the "lesser" factor to consider.

Silly Example

// find some way to call f with 1234
pub fn callWith1234(f: var) void {
    if (!@isCompileError(f(0))) {
        f(1234);
    } else if (!@isCompileError(f(""))) {
        f("1234");
    } else @compileError("f does not take integers or strings");
}

Another Example

pub fn supportsPlus(comptime T: type) bool {
    return !@compileError({ T a; _ = a + a;});
}
pub fn hasAddFieldForItself(comptime T: type) bool {
    return !@compileError({ T a; _ = a.add(a);});
}
// find some way to double the given foo value
pub fn double(foo: var) @typeOf(foo) {
    if (!@compileError({@typeOf(foo) f; _ = f * 2; })
        return foo * 2;
    if (supportsPlus(@typeOf(foo)))
        return foo + foo;
    else if (hasAddFieldForItself(@typeOf(foo)))
        return foo.add(foo);
    else @compileError("don't know how to add foo to itself");
}

More Realistic Example

pub fn isIterable(comptime T: type) bool {
    return !@isCompileError({ T it; while(it.next()) |e| { } });
}

// find some way to get the element at the given index
pub fn getElementAt(x: var, index: usize) {
    if (!@isCompileError(x[index])) {
        return x[index];
    } else if (isIterable(@typeOf(x))) {
        var i : usize = 0;
        while (x.next()) |e| {
            if (i == index)
                return e;
            i += 1;
        }
        return error.IndexOutOfBounds;
    } else @compileError("getElementAt does not support " ++ @typeName(@typeOf(x)));
}

pub fn supportsNull(comptime T: type) bool {
    return !isCompileError({var t: T = null;});
}

// return a variant of the given type T that can be assigned a null value
pub fn NullableType(comptime T: type) type {
    return if (supportsNull(T)) T else ?T;
}

// find some way to return the last element of x
pub fn last(x: var) var {
    if (!@compileError({var : usize = x.len;})) {
        return if (x.len == 0) ? null : getElementAt(x, x.len - 1);
    } else if (isIterable(@typeOf(x)) {
        var lastE : NullableType(@typeOf(x.next())) = null;
        while (x.next()) |e| {
            lastE = e;
        }
        return lastE;
    } else @compileError("don't know how to get last element of " ++ @typeName(@typeOf(x)))
}

[matu3ba]

I would be in favor of a comptime-only growable slice, which only exists in a certain compilation mode and can be accessed with expextCompileError based on comptime-formatted input. Mixing fatal errors with regular user code sounds otherwise like a bad idea.

However, this should likely be deferred until the comptime allocator (interface) exists.

[matu3ba]

If we can spawn processes, then we can parse the output message like implemented in #15991.