andrewbaxter / genemichaels

Even formats macros
ISC License
85 stars 5 forks source link
formatter parser rust

Gene Michaels

Status: Delta (the one after gamma). I've been using it for years without major issue, and I've tested against various code bases and doesn't blow them up. I think other people might use it too! Right now post-formatting pre-writing it re-parses and confirms all comments are consumed as safety checks. Also files over 500kb may take all your memory and invoke the OOM killer.

Named after Gene Michaels.

Everything includes macros and comments. Dog fooded in this repo.

Differences to Rustfmt

Usage

Run cargo install genemichaels.

Running genemichaels will by default format all files in the current package (looking at Cargo.toml in the current directory). You can also pass in a list of filenames to format.

VS Code

If you're using VS Code, add the setting:

  "rust-analyzer.rustfmt.overrideCommand": [
    "${userHome}/.cargo/bin/genemichaels", "--stdin"
  ]

to use it with reckless abandon.

Configuration

The configuration file is optional with (I think) sane defaults if not provided. A default config file named .genemichaels.json in the same directory as Cargo.toml or the current directory if not in a project will be used.

The config contains parameters that tweak the formatting output, suitable for establishing a convention for a project. Things that don't affect the output (thread count, verbosity, etc) are command line arguments instead.

The configuration file is json, but it will strip lines starting with // first if you want to add comments.

Here's the config file. All values shown are defaults and the keys can be omitted if the default works for you.

{
  // Ideal maximum line width. If there's an unbreakable element the line won't be split.
  "max_width": 120,
  // When breaking a child element, also break all parent elements.
  "root_splits": false,
  // Break a `()` or `{}` if it has greater than this number of children.  Set to `null` to
  // disable breaking due to high child counts.
  "split_brace_threshold": 1,
  // Break a `#[]` on a separate line before the element it's associated with.
  "split_attributes": true,
  // Put the `where` clause on a new line.
  "split_where": true,
  // Maximum relative line length for comments (past the comment indentation level). Can be
  // `null` to disable relative wrapping.  If disabled, still wraps at `max_width`.
  "comment_width": 80,
  // If reformatting comments results in an error, abort formatting the document.
  "comment_errors_fatal": false,
  // Genemichaels will replace line breaks with it's own deterministic line breaks.  You can
  // use this to keep extra line breaks (1 will keep up to 1 extra line break) during comment
  // extraction. This is unused during formatting.
  "keep_max_blank_lines": 0
}

Disabling formatting for specific comments

Since comments are assumed to be markdown they will be formatted per markdown rules. To disable this for certain comments, start the comment with //. like

//. fn main() {
//.    println!("hi");
//. }

Disabling formatting for specific files

To skip specific files, in the first 5 lines of the source add a comment containing nogenemichaels, ex:

// nogenemichaels
...

Programmatic usage

Do cargo add genemichaels

There are three main functions:

If you want to format a TokenStream, parse it into an AST with syn::parse2::<syn::File>(token_stream) then call format_ast.

The format functions also return lost comments - comments not formatted/added to the formatted source after processing. In an ideal world this wouldn't exist, but right now comments are added on a case by case basis and not all source tokens support comments.

How it works

At a very high level:

The algorithm basically wraps nodes until all lines are less than the max line width.

This was a simplified explanation; there are a few other factors:

Multi-threading

By default Gene Michaels formats multiple files on all available cores, but this uses proportionally more memory. If you have a project with particularly large files you can restrict to a smaller number of cores in the configuration.

Comments

Comments deserve a special mention since they're handled out of band.

syn doesn't parse comments (except sometimes) so all the comments are extracted at the start of processing. Generally comments are associated with the next syntax element, except for end of line // comments which get associated with the first syntax element on the current line.

When building split groups, if the current syntax element has a token with a line/column matching an extracted comment, the comment is added to the split group.

Macros

Macros are formatted with a couple tricks:

  1. If it parses as rust code (either an expression or statement list), it's formatted normally.
  2. If it doesn't, it's split by ; and , since those are usually separators, then the above is tried for each chunk.
  3. Otherwise each token in the macro is concatenated with spaces (with a couple other per-case tweaks)

Formatting end user use of macros is prioritized over formatting macro_rules, since macros are used more than they're defined. Most macros look like normalish Rust syntax so many of the normal formatting rules can be used.

Q&A

See this Reddit post for many questions and answers.

For other questions or bug reports, etc. please use issues and/or discussions here.