[hook] `hook/generate.dart` 🪝

[dcharkes]

We'd like a standardized way to generate code in Dart packages that does not require a user step.

• Macro's can only take Dart code as input and output.
• package:build_runner requires users to do a manual extra steps

In some hook/generate.dart this would be a package:build_runner equivalent, but it would be aware of the Dart/Flutter SDK and have no race conditions between saving files and running dart and flutter commands (see below).

In https://github.com/dart-lang/sdk/issues/54334, we've discussed multiple aspects w.r.t. code generators. Today we discussed some more requirements offline:

• Automatically running code generators before pub publish to ensure no files are outdated.
• Automatically running code generators after editing dependencies. (Or having the IDE prompt to rerun them.)
• Automatically running code generators after pub get, before dart analyze. (Only needed if the generated code is not checked in.)

Assumptions:

• The generated files are checked in. (Maybe we should aim for not checking generated files in? E.g. it would work more like macros and you'd be able to resolve to the generated files in the IDE, but they are not checked in.)
• Code generators can be heavy to run (definitely longer-running than macros).

Some open questions:

• Should it be a different hook than hook/build.dart? Or a mode for the build hook?
  • If it's the build hook, when do we run this hook in which modes.
  • If it's not the build hook, how do we ensure this generate hook is finished before running the build hook? (E.g. generate is run on-save of its dependencies, like build_runner. But then if you hot-restart in Flutter, you want that on-save action to be fully done first.)

Use cases:

• FFIgen, JNIgen, Swiftgen interop
• translation messages (JSON -> Dart API)

Background knowledge:

• build.rs enables generating of Rust files. https://doc.rust-lang.org/cargo/reference/build-script-examples.html#code-generation

Thanks @mosuem @HosseinYousefi @mkustermann @liamappelbe for the discussion! Please elaborate on things I left out.

[dart-github-bot]

Summary: This issue proposes a standardized way to generate code in Dart packages without requiring user intervention. It aims to address limitations of existing solutions like macros and build_runner, which either lack flexibility or require manual steps. The goal is to automatically run code generators before publishing, after dependency changes, and before analysis, ensuring generated code is up-to-date and consistent.

[mosuem]

Another option would be to include build_runner in the SDK somehow, solving the issue with package:build_runner requires users to do a manual extra steps. This might run against the macros efforts though.

[lrhn]

TL;DR: Either run automatically or check in the code, not both.

Macro's can only take Dart code as input and output.

That's an assumption for the input. We don't currently let macros access the file system, but we could. If we did, then you could pass the URI of a file as an argument in the macro application, have the macro load the file (in a way that can be cached in the macro server between phases) and then it can generate code from that data too.

The alternative is to take the entire file, convert it to a Dart constant value, and have that in the macro annotation, which is just a two-step code generation where you generate the macro application code. (I'd totally do that.)

Assumptions:

• The generated files are checked in.

That's a very strong warning signal for me. Never check in auto-generated code.

It's OK to have a generator that you run manually to generate some code, and then you check in the generated code. It's a code updater then, not a code generator. When people check out the code, the code they see is the code they get, nobody will overwrite it with something else. It's stable.

It's also OK to have an automatically run generator, which generates files on demand, before they are needed. Those files should not be checked in. If the code that is checked in can be changed without you asking for it, then you can't trust the repository code anyway. It's better to have no code then.

If an automatically run generator is for Dart code, macros should be perfect for that. That's exactly the job macros are designed for, as long as they can get the data needed to generate the code. It's automatically generated code, guaranteed up-to-date, that you can still inspect, because every tool that processes Dart programs are aware of macros. It avoids any need ever storing the code, and checking it in.

Now, for non-Dart output, macros are obviously not the answer. There will need to be some code to generate such files. And a standard for where to place them. (Should the be in lib/ if another package depends on them?)

If files are checked in, I'd be more comfortable with a non-automatically run step, where you have to run dart generate to create/update the generated files. There is no need for someone just depending on a Pub packge to run dart generate for that package, if all generated files are checked in anyway.

Code generators can be heavy to run (definitely longer-running than macros).

That doesn't compute for me. Macros are code generators. You may have some heavy use-cases in mind, but whatever framework we create here should be able to handle both small and big tasks. And so should macros.