LLVM rewrite

[madmann91]

This PR tracks the current status of the llvm_rewrite branch.

[leissa]

First examples are working again. Thorin output is currently broken. Since I need this to track down the remaining issues, I started the porting branch which incorporates a number of improvements from t2. As soon as porting is on a par with llvm_rewrite, I'll merge them.

[madmann91]

Let us know if there's something we can do (testing? bug fixing?).

[leissa]

Okay, so I merged (and removed) the porting branch into llvm_rewrite. As a next step, I will implement a textual output that is at least usable to some extent for debugging purposes. Then, I will continue looking into the remaining issues in the llvm backend.

The biggest changes (besides tons of small things here and there) are:

• The Scheduler now only computes a place (a continuation) for each Def in a Scope but not a concrete instruction list.
• The llvm backend recursively invokes emit and places a new llvm::Value where necessary.
• For this to work, each Continuation/basicblock in a scope is associated its own llvm::IRBuilder.
• This is why the irbuilder_ member is gone and the current irbuilder is instead passed to a ton of methods.
• I've already done that in the core of the llvm backend
• I've not yet ported the other llvm-related backends like the NVVM or AMDGPU backend (the files are commented in the CMakeLists.txt right now).
• This should be mostly boilerplate and is definitely sth one of you guys can continue working one while I delve into the depths of the llvm backend.

Regarding the C-Backend: As promised, this needs some major overhaul ;) We would need to do the same adjustments as in the LLVM backend here. What is more, since the C backend is currently broken anyway, I changed the streaming API (from the t2 branch) into a more std::fmt kind of style. Actually resorting to std::fmt (or the fmt library) would definitely reduce some boilerplate on our side while gaining more features, but so far I didn't have the patience to do that.

[madmann91]

Alright, I'll have a look at the things that need porting right now. I think I've already mostly ported the NVVM stuff anyway.

[madmann91]

I think the remaining three backends rely on the C backend to work, so I'll leave them commented out until the C backend is done.

[leissa]

Another issue you might look into is the split_slots phase. It also relied on the instruction list - which is non-existent anymore. We need to re-implement this to simply work recursively.

[madmann91]

We can simply leave it for the time being. It's only relevant for the C backends anyway, since LLVM has the "scalar replication of aggregates" phase. I've created a branch in artic named llvm_rewrite that contains fixes for this branch of thorin. There are still bugs before even getting into the LLVM backend. The following piece of code produces an assertion with artic on llvm_rewrite:

fn test2(x: i32, _: i32) -> i32 {
    test1(x, 2)
}
fn test1(x: i32, _: i32) -> i32 {
    test2(x, 1)
}

The result is the following:

> bin/artic test.art --emit-thorin -O1
artic: /home/madmann/Projects/anydsl/thorin/src/thorin/def.cpp:38: void thorin::Def::set_op(size_t, const thorin::Def*): Assertion `def && "setting null pointer"' failed.
fish: “bin/artic test.art --emit-thori…” terminated by signal SIGABRT (Abort)

[leissa]

I'm looking into it.

[leissa]

Your last example isn't a regression but is in fact also broken on master. I fixed it in 120349df6fff9fb889dc412a1239dc49de546e49 and ported to llvm_rewrite.

[leissa]

Most impala examples work again save 2-3 test cases. However, there are still some messy thing's I'd like to improve.

• I want to wire the phis in a more clever way that we don't need Param::peek anymore. This thing will also get pretty messy as soon as we switch to tuples instead of arg-lists.
• I want to polish the output a bit more.

[leissa]

All except one example from impala work now. I will investigate the remaining example tomorrow. Then, I'll nuke Param::peek as mentioned above. Then, we can discuss any remaining issues and what to do with the C backend.

[leissa]

Alright, I think I'm done. I will polish the Thorin output a bit. But this is orthogonal to the other stuff. So the elephant in the room is: What will happen with the C backend?

[madmann91]

Well we should port it I suppose?

[leissa]

:D The question was more, who is going to do that. The problem I have with the C backend, that there was never an easy way to test this thing.

[madmann91]

You could just use Stincilla in OpenCL mode, with pocl

[leissa]

I will do that.

[leissa]

Okay, I looked into libfmt as an alternative to what we currently have. But there are a couple of serious problems that don't make libfmt a good fit for us. So, I'll stick with the streaming API we currently have in llvm_rewrite:

• It's only 150 LoC
• super easy to use
• even better support for streaming ranges (libfmt wouldn't work in our case anyway as libfmt doesn't like pointers at all)
• no dependencies issues
• support for indentation even when streaming ranges that use \n as seperator.

The downside is that we don't have this super fancy format mini language. If we really need that for whatever reason, we could still use libfmt for these cases.

[madmann91]

Another downside is that std::fmt is part of the standard in C++20, which means we would have this functionality without having to rewrite it once it's "old enough" to be implemented by all compilers.

[leissa]

As I argued above, std::fmt doesn't help us. It's basically the same as libfmt and there are a couple of serious problems that don't make libfmt/std::fmt a good fit for us. So std::fmt/libfmt is at best a nice-to-have addition.

[leissa]

My last commit fixes the real problem of "bug fix: only emit phi-args for valid args". We must be careful to not forget the else-branch for cases like this:

if (auto sth = emit_unsafe(asdf); !sth.empty()) {
    // ...
} else {
    return "";
}

[leissa]

I added to impala's test script that the C backend will be tested as well. So far we are passing 71/100 tests (8 of which are inline asm examples). One probl is that we have to either run a topo sort to emit all functions in dependency order, or we have to declare them beforehand (and remove static). Example:

static void foo() {
    bar();
}

static void bar() {}

[madmann91]

This should not be a problem. We should be emitting them as prototypes. I am working for a fix that emits every function prototype, but there are a couple of things that need refactoring first.