In other words, "let's become self-hosted". :slightly_smiling_face:
With the effort done in #406, this is no longer a distant pipe dream. Here's the rough plan, as it currently stands:
Start rewriting some of the simpler parts of the compiler in Perlang, perhaps beginning with a simple method or so:
Write it in Perlang
Add a mechanism in the Perlang compiler for compiling a given .per file to .cc and/or .o file, without producing a full executable. We could even add a --lib flag or something to indicate that we want to generate a library, but aggregating multiple .per files into a single .cc file if that's simplest for now could also be fine (note, will require generating the method definitions at the head of the file because of the way C++ works).
Use the existing Perlang compiler snapshot to compile its code to C++ (note: we'll probably need to "lock" the Perlang snapshot used here to a specific Perlang revision, to avoid surprises)
Given the "end goal" described below, we could even skip the "Use the existing Perlang compiler to compile its code"; we could just "build the code" and use that to regenerate the C++ parts, couldn't we? That way, there'll be no annoying binary snapshots to take care of etc.
Add the generated C++ code to the repo (in other words, commit the transpiled code as well for now)
Find a way to expose this C++ code to C#, using P/Invoke
Iterate on this for a while, rewriting more and more "simple" parts in Perlang. Try to find some low-hanging fruit to begin with, to get a general feeling of the feasibility of this.
Challenges
Some parts will be reasonable to rewrite, like the ones described above, but other parts of the compiler will provide a significantly more challenging experience. I am thinking about the scanning, parsing and tree-walking parts here. Those are all "glued together" because of the data structures they use (C# classes to represent the different expression and statement types). We can either:
Do it as a "big-bang" conversion, where we convert all relevant code to Perlang in one big PR. Not a great idea, and it will be a huge undertaking to try to do it like this. And because of the immaturity of the language as it stands, we might even be sidetracked during the process by wanting/needing to add more language functionality to support the effort, delaying it even further.
Find a way to "inter-op" between C# and Perlang.
The latter seems "more doable" in a way, but then again:
It's not trivial either, and
It's a dead-end which will only be needed to achieve this epic. Once we are done, what will be the usefulness of inter-op:ing between C# and Perlang? Sure, it might be nice but it's currently considered a "non-goal" for the project at large.
Let's postpone this for now, but it's still good to get this noted down here.
Timeframe
My current estimate is that the full rewrite of the compiler into Perlang will take about 3-5 years, given our current capacity. The process will hopefully help us mature the language and the compiler as a nice side-effect.
Use the C version of the V compiler to build a functional compiler, without the V compiler.
Then, recompile the V source code using the generated compiler in step 1.
Finally, recompile the V source code using the built-from-V compiler in step 2. :slightly_smiling_face:
All of this can be seen in their simple Makefile. We could opt for something similar: the end goal could be to maintain the Perlang compiler as a single C++ file, "all-inclusive", only depending on the Perlang stdlib (also written in C++), which is then statically linked with the compiler => produces a native Perlang compiler for an arbitrary platform.
Then, we use that compiler to regenerate our C++ sources, just like our friends in the V project does it. :grin: Wouldn't that be quite nice?
Moved to GitLab
Please continue to the new version of this issue here: https://gitlab.perlang.org/perlang/perlang/-/issues/454. The GitHub page you are currently reading will not contain the latest information on this issue.
In other words, "let's become self-hosted". :slightly_smiling_face:
With the effort done in #406, this is no longer a distant pipe dream. Here's the rough plan, as it currently stands:
.perfile to.ccand/or.ofile, without producing a full executable. We could even add a--libflag or something to indicate that we want to generate a library, but aggregating multiple.perfiles into a single.ccfile if that's simplest for now could also be fine (note, will require generating the method definitions at the head of the file because of the way C++ works).Challenges
Some parts will be reasonable to rewrite, like the ones described above, but other parts of the compiler will provide a significantly more challenging experience. I am thinking about the scanning, parsing and tree-walking parts here. Those are all "glued together" because of the data structures they use (C# classes to represent the different expression and statement types). We can either:
The latter seems "more doable" in a way, but then again:
Let's postpone this for now, but it's still good to get this noted down here.
Timeframe
My current estimate is that the full rewrite of the compiler into Perlang will take about 3-5 years, given our current capacity. The process will hopefully help us mature the language and the compiler as a nice side-effect.
End-goal
The The V Programming Language has done their V-to-C transpiler in an interesting way. They provide the transpiled sources in an auto-generated repo here: https://github.com/vlang/vc.
That way, you can:
All of this can be seen in their simple
Makefile. We could opt for something similar: the end goal could be to maintain the Perlang compiler as a single C++ file, "all-inclusive", only depending on the Perlang stdlib (also written in C++), which is then statically linked with the compiler => produces a native Perlang compiler for an arbitrary platform.Then, we use that compiler to regenerate our C++ sources, just like our friends in the V project does it. :grin: Wouldn't that be quite nice?