Improve reliability of finding the exact location of ripgrep's matches

acheronfail / repgrep

An interactive replacer for ripgrep that makes it easy to find and replace across files on the command line.

Apache License 2.0

267 stars 4 forks source link

Improve reliability of finding the exact location of ripgrep's matches #11

Closed acheronfail closed 4 years ago

acheronfail commented 4 years ago

As discussed https://github.com/BurntSushi/ripgrep/issues/1627#issuecomment-649858419.

Rather than trusting the absolute_offset and start + end items in the match JSON object provided by ripgrep, we should instead use the same regex engine that ripgrep uses, use the line_number provided and try to perform our own search of the line to discover the exact location.

Potentially useful links:

https://docs.rs/grep/0.2.7/grep/

acheronfail commented 4 years ago

I tried to test this out, and found that the structs that the grep crate gives us still don't contain the byte location information (see fix/proper-encoding-support branch).

Asked in ripgrep:

So, if we want to do this via the grep crate we will have to wait for the above to be fixed.

We may be able to do some manual matches via the regex::bytes export from the regex crate itself...

acheronfail commented 4 years ago

After a little research and tests, I found this approach to be both the simplest to implement, and reliable enough for our needs:

open the file and detect its encoding
decode the file into a String (UTF8)
use ripgrep's indices and match ranges to replace portions in this string
re-write the file to disk, re-encoding it back into its source encoding (re-inserted a BOM if one was found)

This is the way ripgrep searches for matches most of the time. It's also the way VSCode opens and edits files too. I think performing all our replacements on a format that's native to Rust is the most maintainable way forward.