search

oxc-project / oxc

⚓ A collection of JavaScript tools written in Rust.
https://oxc.rs
MIT License
12.22k stars 442 forks source link

Rework `oxc_prettier` #5068

Open Boshen opened 2 months ago

Boshen commented 2 months ago

crates/oxc_prettier was my attempt at the prettier bounty.

I thought I could finish it in time, except the fact that I rushed too quickly without looking at all the requirements ... It was too late when I got blocked by printing comments.

In order to rework oxc_prettier, we need to understand at least:

As for the infrastructure, we already have most of the code:

Feel free to remove everything and start from scratch, and copy over the format code https://github.com/oxc-project/oxc/tree/main/crates/oxc_prettier/src/format

parkin-lin commented 2 months ago

I'm sorry to take up some of your time, but I feel confused about the following issues, so please allow me to ask you a few questions... As we konw, the next minor release of ESLint will deprecate core formatting rules. But this also means that the linter itself originally had code formatting capabilities, so why isn't Oxlint directly compatible with plugins like ESLint Stylistic? Are there some corner cases where Oxlint doesn't work as well as Prettier, so Oxlint had to be compatible with ESLint and Prettier separately, or it's just a divide-and-conquer engineering trade-off？ I'm guessing one of the reasons for this is that Prettier can support different file formats (not only .js, but also .css and more...), but ESLint or Oxlint can do the same job via plugins. Thanks.

DonIsaac commented 2 months ago

We made an intentional decision to only support code correctness rules, not stylistic rules. @Boshen can explain reasoning more.

Boshen commented 2 months ago

@leaysgur is writing a series of articles in preparation of this task:

I'm also working on comment attachments to unblock prettier.

leaysgur commented 1 month ago

I'm also working on comment attachments to unblock prettier.

Does this mean that you aim to implement Prettier equivalent which achieves with 60+ utils? 🫨

FYI: Babel also has relevant code(just 300 loc), but that did not seem to meet Prettier's requirement.

Boshen commented 1 month ago

Does this mean that you aim to implement Prettier equivalent which achieves with 60+ utils? 🫨

We need to figure out what exactly is prettier doing with those 60+ utils 🥲

IWANABETHATGUY commented 1 month ago

For those who are interested in algorithms under the hood, prettier is based on https://homepages.inf.ed.ac.uk/wadler/papers/prettier/prettier.pdf, https://prettier.io/docs/en/technical-details

leaysgur commented 1 month ago

It has been 3 weeks since I started reading the Prettier source code. It's still far from being complete, but I'd like to leave some progress and summary here.

How to debug Prettier

There are 3 ways:

https://leaysgur.github.io/posts/2024/09/03/111109/

It is written in Japanese, but it is all code, so you can understand it. 😉

I also recommend to run node --inspect-brk the code with debugger and inspect it from Chrome DevTools.

How to handle comments

I will post some topics for discussion in a few days.

leaysgur commented 1 month ago

How to handle comments

As you may know, Prettier's formatting process consists of roughly 3 phases:

Comments are collected in P1 and used in P2.

In P1:

As a result, some AST nodes have comments property with array of Comment extended with leading, trailing and few more props.

In P2 (I haven’t read the code in detail here yet),

In OXC, part of the necessary information is already implemented and can be obtained. / #5785

However, just like with Babel, that information may be different from what Prettier requires...

So, I think I’ve generally understood "what" Prettier is doing.

However, as for "why" Prettier does it that way, I can only say it’s because that’s Prettier’s opinion.

Incidentally, there seem to be at least around 120 issues related to JS/TS at the moment, but

https://github.com/prettier/prettier/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen+label%3Alang%3Ajavascript%2Cjsx%2Ctypescript+label%3Atype%3Abug

about 50 of them are related to comments, with some remaining unresolved since as far back as 2017.

https://github.com/prettier/prettier/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen+label%3Alang%3Ajavascript%2Cjsx%2Ctypescript+label%3Atype%3Abug+label%3Aarea%3Acomments

leaysgur commented 1 month ago

So, what I would like to discuss(confirm?) is: where should oxc_prettier aim to go? As the name suggests, should it strive for 100% compatibility with Prettier?

If the goal is to achieve compatibility, I think it would mean porting almost all of Prettier’s lines.

I believe the original aim of this issue was this, but again, is this an acceptable?

If we decide to proceed, let’s consider how we might specifically approach it...

Fortunately, the 3 phases have isolated inputs and outputs, so:

I think reducing the gaps between each phase will be important.

For that, we’ll need some way to directly compare the result of Prettier.parse() with our implementation.

// OXC
let oxcASTWithComments = oxc_prettier::parse(text);
let oxcEstreeASTWithComments = estreeJSON(oxcASTWithComments);
// let oxcBabelASTWithComments = babelJSON(oxcASTWithComments);

// Prettier
let prettierEstreeASTWithComments = Prettier.parse(text, { parser: "meriyah" });
// let prettierBabelASTWithComments = Prettier.parse(text, { parser: "babel" });

// Diff!
assert(oxcEstreeASTWithComments, prettierEstreeASTWithComments);

What if we don’t aim for full compatibility?

But in that case, maybe it would be more like oxformat rather than oxc_prettier?

Anyway, this is what I was thinking these days. What do you think?

Boshen commented 1 month ago

For the long run, I envision a more configurable and less opinionated formatter.

Under this assumption, we can relax the "100% compatibility" constraint.

My intention is to reach a high compatibility with prettier so we can bootstrap ourself, and then start deviating behavior.

I've also looked at the prettier and rustfmt issues before, comment positioning is a really difficult subject due to the natural of matching the original intent of the comment.

To move things forward, I suggest @leaysgur to start refactoring our prettier code in preparation for what to come, I believe you know more about prettier than I do. You may ask @Sysix for help given they have started working on this area as well.

Our code for P2 and P3 is also incomplete, I suggest to do a little bit of rework first, as well as adding more debug utilities.

Boshen commented 1 month ago

I think reducing the gaps between each phase will be important. For that, we’ll need some way to directly compare the result of Prettier.parse() with our implementation.

I don't think we need to do this. This will leave us too coupled with the prettier implementation, and it may end up being a waste of effort.

We are already matching half of the generated text, a few more iterations of refactoring and implementing details should close our gap to prettier.

leaysgur commented 1 month ago

I see, what I likely concerned about was: what exactly is meant by "high compatibility". 😅 Does it have to be 100%, or should aim for at least 80% if possible, or is something around 70% good enough? etc...

So, It's good to know that we're not necessarily aiming for 100% compatibility.

It's still uncertain how much the percentage will drop if we give up on porting completeness for comment handling, but for now, I'll move forward.

Following your suggestion:

I’ll work on these whenever I have time! 🚀

For the long run, I envision a more configurable and less opinionated formatter.

I love this idea!

magic-akari commented 1 month ago

For the long run, I envision a more configurable and less opinionated formatter.

Does this mean that oxc_prettier will provide a wide range of configurable options, and offer a preset called prettier to match with prettier?

leaysgur commented 1 month ago

How to handle comments

(Follow up of https://github.com/oxc-project/oxc/issues/5068#issuecomment-2372777159)

As I posted above, comments are collected and attached to AST nodes in P1. Then in P2, comments are printed as Doc along with other AST nodes.

Most comments are printed with their attached nodes like:

[leadingCommentsDoc, nodeDoc]
// or
[nodeDoc, trailingCommentsDoc]

https://github.com/prettier/prettier/blob/e3b8a16c7fa247db02c483fb86fc2049d45763b8/src/main/ast-to-doc.js#L128-L138

But the rest of the comments are handled as needed.

There are about 40 files for printing ESTree AST to Doc.

https://github.com/prettier/prettier/tree/main/src/language-js/print

And 15 files of them print comments on demand.

❯ rg 'print(Dangling)?Comments(Separately)?' -l src/language-js/print
estree.js
class.js
type-annotation.js
function-parameters.js
mapped-type.js
component.js
module.js
function.js
ternary.js
array.js
binaryish.js
property.js
call-arguments.js
block.js
jsx.js
arrow-function.js
object.js
member-chain.js
type-parameters.js
Boshen commented 1 month ago

@leaysgur I was just informed that biome_formatter is a port of the Prettier IR and printer infrastructure. It may be possible to reduce repeated work by using biome_formatter. More investigation for the rework!

So instead of using our current unfinished IR and printer, we can replace them with biome_formatter.

I pinged you on discord, for full context.

leaysgur commented 1 month ago

Thanks for the pinning!

biome_formatter is a port of the Prettier IR and printer infrastructure.

Yes, and I just started reading through biome_formatter and biome_js_formatter, so the timing is perfect. 😄

I hadn’t considered it as a viable option for OXC (though I’m not entirely sure why), but if we shift our goal to generating biome_formatter's IR, it could significantly reduce the amount of work, maybe?

I’ll look into this soon (after finishing the regex parser rework), including:

Boshen commented 1 month ago

but if we shift our goal to generating biome_formatter's IR

biome_formatter IR, aka format_element

I think it's the same thing as prettiers IR. Also Doc in our code.

srijan-paul commented 2 weeks ago

Hey all, just dropping by!

My intention is to reach a high compatibility with prettier so we can bootstrap ourself, and then start deviating behavior.

I feel adding support for comments to the current implementation should be feasible. In fact, I suspect it'll give us a significant boost on the prettier conformance test suite from the current 39%. We don't have to follow prettier's implementation exactly, but we can do something close to it.

Since the oxc_ast's Program node already stores a pub comments: Vec<'a, Comment> field, we could just repurpose it to add a third pass over the Doc objects (before the Commands are generated)? I think it'd end up looking similar to what @leaysgur showed here.

I could do a PoC on my fork and have it up in some time. Would any of you be interested in taking a look, perhaps?

I don't know how much more work the parenthesis bit will take, but my hunch is that emitting biome IR might be more work than supporting comments in oxc_prettier's current form. Of course, people who've worked on this will know better. Curious to know what you think!

pumano commented 2 weeks ago

I want to share my overall idea from end user perspective: After oxc_prettier will be created I suggest to create something like eslint-plugin-oxc-prettier in future for using oxlint with that plugin to show problems in IDE / console and fix it as eslint-plugin-prettier does.

Currently my lint CI time is 50% time for prettier eslint (too much).

YurySolovyov commented 2 weeks ago

you might want to consider running prettier (or oxc_prettier later) separately: https://www.joshuakgoldberg.com/blog/you-probably-dont-need-eslint-config-prettier-or-eslint-plugin-prettier/