search

kpdecker / jsdiff

A javascript text differencing implementation.
BSD 3-Clause "New" or "Revised" License
7.69k stars 491 forks source link

Fix diffWords handling of whitespace #497

Closed ExplodingCabbage closed 4 months ago

ExplodingCabbage commented 4 months ago

This resolves https://github.com/kpdecker/jsdiff/issues/436.

Consider the dramatic example of using diffWords to diff this text...

FOUR FIVE SIX SEVEN EIGHT NINE

against

ONE TWO THREE FOUR FIVE SIX

The diff we intuitively expect to get is one where we insert the three words ONE TWO THREE at the beginning, keep the three-word phrase FOUR FIVE SIX that's common to both texts, then delete the three words SEVEN EIGHT NINE from the end.

On master as it currently exists, though, we instead get this absurd and unintuitive diff, where ALL SIX WORDS from the original text get deleted and all six words from the new text get inserted. Thus we delete the words "FOUR", "FIVE", and "SIX" and then reinsert them.

image

Why this absurd result? Because on master, runs of whitespace are also tokens, and preserving one of those is worth as much to the score of a diff as preserving an actual word. The six extra insertions/deletions of words necessitated by not preserving any words saves us from having to insert or delete six spaces, so is considered just as good an outcome!

On this branch, we stop treating spaces as tokens, and therefore get this much more sensible diff:

image

It's annoyingly complicated. But IMO this should convert diffWords from basically being a horrible trap for users that you should probably never use for anything, into providing pretty decent word-level diffs most of the time (albeit with some slightly weird handling of whitespace changes). IMO this is a definite improvement.

ExplodingCabbage commented 4 months ago

Okay, I reckon this is ready to merge. Dunno if anyone is paying attention to what I'm doing but I'll leave this open until Monday just in case someone wants to comment!