[Feature Request] Syntactic Macros and First-Class Metaprogramming in Mojo

[ParadaCarleton]

Review Mojo's priorities

• [X] I have read the roadmap and priorities and I believe this request falls within the priorities.

What is your request?

First-class metaprogramming with syntactic macros (which execute at compile time).

What is your motivation for this change?

Syntactic macros are extremely useful for:
1. Rewriting nice-looking code into ugly-looking but high-performance code, without the need for users to interact with the ugly code.
2. Creating DSLs.

Any other details?

No response

[mojodojodev]

It already supports metaprogramming: https://docs.modular.com/mojo/programming-manual.html#parameterization-compile-time-metaprogramming

It uses the language itself to run code at compile time like Zig, instead of a separate macro language like Rust. There are some extra features that push it further as well, you can invoke compile time functions that try out different implementations to test what's the fastest on the target it's compiling to: https://docs.modular.com/mojo/MojoStdlib/Autotune.html. You can also use any function at runtime or compile time, you don't have to mark it const fn like Rust or constexpr like C++

[ParadaCarleton]

It already supports metaprogramming: docs.modular.com/mojo/programming-manual.html#parameterization-compile-time-metaprogramming

I'm not familiar with Zig, but unless I'm misunderstanding something, this looks like a very different set of features from what I meant or would consider first-class metaprogramming. You can look at Lisp, Nim, or Julia for examples of what I'm talking about.

[ksandvik]

What I understood https://docs.modular.com/mojo/programming-manual.html#parameterization-compile-time-metaprogramming is compile time, similar to Nim. This all is kind of new and might change, but what are the specific features you are looking for?

[lattner]

As mentioned above, compile time metaprogramming is a major feature for Mojo and we'll continue to invest in it. I think it is important to have some sort of macro-like feature for syntax - one specific use-case for this is to enable things like "parallel_for" to be library features instead of language features.

That said, we aren't looking to implement language features for their own sake, we should be motivated by specific problems we want to solve. I'm going to close this now to tidy the tracker, not because we won't do this, but because we'll solve individual problems as they come up, thanks!

[ParadaCarleton]

What I understood docs.modular.com/mojo/programming-manual.html#parameterization-compile-time-metaprogramming is compile time, similar to Nim. This all is kind of new and might change, but what are the specific features you are looking for?

Some major missing features (apologies if I missed something in the Mojo docs!):

• Homoiconicity (code is data, with its own data structure)
• Hygienic macros (by default, macros do not interact unexpectedly with user-defined variables)
• Multiple stages of compilation/metaprogramming (e.g. Julia's macros run right after parsing, and generated functions run right after type inference)

You can find some good descriptions of these key features in the documentation for Elixir or Julia.

Mojo's autotune would be a good example of something you can do with syntactic macros; I know there's similar optimizers in Julia. Particularly in AI research, these are very important for researchers to create their own code-optimizing code, i.e. their own implementations of autotune-like algorithms. Most Python libraries behave similarly (using Python's runtime as a "compiler" by taking an input, parsing it into an AST to modify it, and then producing optimized code).

[ParadaCarleton]

That said, we aren't looking to implement language features for their own sake, we should be motivated by specific problems we want to solve. I'm going to close this now to tidy the tracker, not because we won't do this, but because we'll solve individual problems as they come up, thanks!

Just wanted to make a quick note--while it doesn't have to be done right away, I do think that if there is interest in this feature, it's important to tackle early on. Macros and metaprogramming are one place where technical debt can pile up very quickly, because there are many different opportunities for code to interact with each other.

[ceving]

It is definitely an ignorant joke, that this has been closed as "completed"! In fact it has been closed as "won't fix".