Structural Search Replace

[matklad]

https://github.com/rust-analyzer/rust-analyzer/pull/2266 added a new function for Source:

impl<T> Source<T> {
    pub fn new(file_id: HirFileId, ast: T) -> Source<T> {
        Source { file_id, ast }
    }
}

The goal of this issue is to replace all Source { file_id: foo, ast: bar } literals with more succinct calls to Source::new(foo, bar).

However, we are not going to do this in a boring manual way. Instead, what we'll do is we provide an IDE feature, such that you enter

Source { file_id: $file_id:expr, ast: $ast:expr }  ==>> Source::new($file_id, $ast)

and IDE uses this template to fix all files.

This is called SSR.

Here's the rough plan:

• add Analysis::structural_search_replace(&self, template: &str) -> Cancelable<Result<SourceChange, SomeError>> method: this will be the main entry point.
• add ssr module to ra_ide_api crate
• implement template parsing:
  • split the input request on ==>> into pattern and template (separator is bikesheddable)
  • using text search (regex or, better in this case I think, hand-rolled) to replace $ident:fragment with __search_pattern_ident
  • parse both pattern and template as expression (we might add more types in the future, but let's start with expressions)
  • Group all the info above into some nice data structures, like SsrPattern and SsrTemplate
• implement search in a single file with SsrPattern. Note that this should do only searching, replacement should be separate. Again, this should return a structured result like SsrMatches.
  • searching means visiting files's AST, and matching each node against pattern. Matching should ignore whitespace difference. Bonus points if it an ignore insignificant order difference (like the order of fields in the struct literal)
• Implement replacement in a single file, which takes a file, an SsrMatches, an SsrTemplate, and produces a `TextEdit
• Implement the driver, which visits all files in the current project and builds a SourceEdit
• Expose this functionality via ra_cli
• Expose this functionality via custom LSP request
• Write typescript code to use this request from VS Code (there's an API to ask user for a string, that should be enough. Alternatively, allow user to write template in a file, and get it from selection )

EDIT: Zulip thread for discussion: https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/Structural.20Search.20Replace

[matklad]

Future work:

• instead of a static template, allow using arbitrary program for replacement, probably by bundling WASM interpreter

[fkohlgrueber]

I'd like to work on this! I'll see how far I can get and come back here if I've got questions.

[bjorn3]

There is already a program which kind of does this. However it is nightly only. https://github.com/google/rerast/

Rerast is a search/replace tool for Rust code using rules. [...] Matching is done on syntax, not on text, so formatting doesn't matter. Placeholders are typed and must match the type found in the code for the rule to apply.

[matklad]

@bjorn3 yeah, I am aware that rerast exist. I believe, however, that, in the limit, rust-analyzer is a significantly better foundation for these kinds of tools.

@fkohlgrueber cool! I want not note, however, that at the moment I am pretty swamped with many things, so I might be slow to review the code. Will happily answer any questions though!

[mikhail-m1]

@fkohlgrueber are you still working on it?

[fkohlgrueber]

@mikhail-m1 unfortunately not. I also didn't make significant progress, so you can take this issue if you want.

[mikhail-m1]

ok, let's see what I can do

[mikhail-m1]

i think this one can be closed, we have #3186 for follow up

[RReverser]

I just tried it for the first time, and it seems that it can't match macro invocations. Is it because it operates on the expanded code rather than source one?

[matklad]

replied at https://github.com/rust-analyzer/rust-analyzer/issues/3186#issuecomment-615334347, as that's indeed the current umbrella issue for ssr.