Potential frontend improvements

[fuhsnn]

Hi, I maintain another fork of chibicc that had gone through plenty of fixes that probably duplicate what you're doing a bit, like your latest commit to support the unary positive sign.

I put the more notable compilation bugs found with chibicc under the upstream tag with minimal repro and attempted to associate the fixing commit with them, though it could be superseded by later refactoring.

I generally don't make compatibility fixes into issues, though you can spot them in relevant tests. To run individual tests do $CC test/__.c -xc test/common.

Just want to reduce some duplicated efforts in the world. And thank you for the test compilation!

[rochus-keller]

Thanks for the hint. I was actually looking for a chibicc fork with fixes, but wasn't able to identify one (too many results, impossible to identify the correct one up-front).

I very much appreciate your upstream tag repo and will check whether I found everything.

Have you by chance been able to successfully compile and run the Lua 5.1 engine with your fork? Here is a condensed set of files which can just be compiled with "cc *.c -lm": http://software.rochus-keller.ch/lua_5.1.5_incl_bit_module.tar.gz The original version seems to segfault.

Yet another challenge is the C99 version of the Are-we-fast-yet benchmark suite, which I generated with my Oberon+ compiler: http://software.rochus-keller.ch/Are-we-fast-yet_ObxIDE_Cgen_2021-12-30.zip Chibicc fails to correctly parse this code; I didn't find a fix yet.

I will do checks with your code and see what I can reuse (I'm still on x86 which makes testing of x86_64 a bit challenging).

[rochus-keller]

I just compiled slimcc und built the are-we-fast-yet code base using the includes of dietlib. And I'm very amazed that it indeed worked (I had to add two include files because of "implicit declaration of a function" errors).

From that I conclude that your version of chibicc is much better than what I used so far. I will try to apply my ecc specific changes on your slimcc and run my ecc testcases with it.

[fuhsnn]

Haven't tried Lua, will check it later!

I'm curious why you're working on a frontend for Eigen, from the website it looks like it already got a C++ parser.

[rochus-keller]

from the website it looks like it already got a C++ parser.

The C++ frontend is in a very early stage; from my humble point of view, implementing a C++ compiler takes a life time; till then we can use my chibicc version to make much existing code (such as the Lua VM) available for testing and performance measurement. It's a good exercise to familiarize myself with the ECS backend and the IR that I actually want to use for my Micron programming language.

Just made a diff between slimcc and chibicc and started a branch which uses your code instead of my chibicc version, besices the codegen.c/cpp; but as it seems slimcc is too far away from chibicc to integrate with my codegen; it's likely less work to use your codegen and start again modifying it so it generates Eigen IR; both ways are out of reach for me at the moment, so I will just apply some of your https://github.com/fuhsnn/slimcc/issues?page=1&q=is%3Aissue+is%3Aclosed+label%3Aupstream to my existing code base.

[fuhsnn]

Most stuff in my codegen.c were attempts to do simple optimization on the AST, I believe all stuff in gen_*_opt() can be left out without issue.

Let me review the history a bit, I'll summarize my changes that are more relevant to you.

[rochus-keller]

That's very kind, thank you very much. I already applied a few of your changes to the preprocessor and will now test.

[rochus-keller]

I had to undo the changes for https://github.com/fuhsnn/slimcc/issues/1 because compiling the Lua 5.1 VM failed with a "too many files open" error.

[fuhsnn]

Sorry for the rant-y style.

Stack layout:

| (over-alignment gap) | local stack | temp stack | (VLA space) | (func call stack-args) | - over-alignment gap: - - if any local variable needs over-alignment, a second stack pointer is used and aligned to the largest align-requirement, this creates a gap - local stack: - - optimization: local variables are made scope-bound (chibicc uses a big list for the entire function), this allows variables of parallel (sibling) scopes be mapped to the same space (see the diff in assign_lvar_offsets()) - - - remember to disable the opt for setjmp() callers - temp stack: - - looks like Eigen manages push/poping, so you can ignore most stuff around my push_tmpstack() - func call stack args - - looks like Eigen manages func-arg pushing, Just know that I changed the linked list of function parameters from Type* to Obj*, and replaced push_args() to assignment AST nodes, a lot of diff noise around this Bit fields: - plenty of bugs with bitfields happend, issues/tests covered them. Switch: - chibicc had trouble 64-bit values, issues/tests covered them case: ND_MEMBER in gen_addr(): - chibicc didn't covered rare patterns like ({struct;}).mem, this is where that's fixed Variadic functions: - chibicc's variadic ABI was wrong, I changed the header implementation to AST nodes, Constant Folding: - most of it actually happens at ND_CAST generation, new_cast() is where it's at, I found it pretty neat and reduce AST nodes allocated

[fuhsnn]

http://software.rochus-keller.ch/lua_5.1.5_incl_bit_module.tar.gz

What lua script did you test with this one? Upstream chibicc on Debian seems to build without problem... In fact it builds Lua 5.4.7 and pass its ./testes nicely.

I had to undo the changes

Could be some implied behavior from related code not merged. There are unfortunately plenty of hidden dependencies between commits, originated from this style of scattering low level operations around to minimize abstraction.

[rochus-keller]

Thank you very much for the summary of your changes. I will check which of your changes I should implement.

Concerning the Lua script: my test horse is https://github.com/rochus-keller/Are-we-fast-yet/tree/main/Lua, but I didn't investigate the issue; could also be that I did something wrong. I'm currently more concerned with Lua running on my ECC version, which is not even capable of doing something simple like "a = 10".

But yesterday I finally found a work-around how to compile my Awfy C version generated by Oberon+; I had to change Oberon+ so it uses more parenthesis; the chibicc parser failed e.g. with

temp = &(T){}[0]

Now it generates

temp = &((T){})[0]

which the parser accepts.

[fuhsnn]

Can't reproduce the Lua segfault, could it simply be unoptimized recursive calls blowup the stack? I've seen that a lot before adding the scope-stack optimization.

the chibicc parser failed e.g. with

I remember that one, just move the compound literal parser from postfix() to primary(). https://github.com/fuhsnn/slimcc/commit/bf3618694a5e20bd44eb13e6998b401277cc5de5

[rochus-keller]

Thanks for the hint.

Instead of investing more time in the chibicc based compiler, I currently consider to switch to cparser and libfirm (either directly generating ECS IR from cparser, or keep the libfirm optimizer and generate ECS from libfirm IR).

[fuhsnn]

I had a brief look at it, libfirm IR being sea-of-nodes is probably a lot harder to manipulate than the chibicc AST. I've been drafting an SSA backend in mind as well... anyway, the more of them the better, test files too!

[fuhsnn]

I made a cleaned-up branch at widcc, just a heads-up in case you're working on chibicc-based stuff.

[rochus-keller]

Wow, great, thank you very much! I will try to combine it with my code generator as soon as I have time and see whether I can make it work.

Meanwhile I make progress in finding out how libfirm and cparser work (see https://github.com/rochus-keller/EiGen/tree/firmc/cparser) and currently I try to make an Eigen backend for libfirm (not yet committed).

[rochus-keller]

I noticed that you're currently working on the code. Do you think it is stable enough that I can try to integrate it with my ecc version, or should I wait with this?

[fuhsnn]

I noticed that you're currently working on the code. Do you think it is stable enough that I can try to integrate it with my ecc version, or should I wait with this?

I'm working on a regression in building Tinycc. It's a x87 long double stuff which shouldn't affect ecc.