search

mmanela / diffplex

DiffPlex is Netstandard 1.0+ C# library to generate textual diffs.
Apache License 2.0
1.01k stars 184 forks source link

fixing "Differ" so that it doesn't generate so crazy amount of tokens… #43

Closed gantrior closed 5 years ago

gantrior commented 5 years ago

The problem can be demonstrated by diffing two outputs of following powershell script: 

(Get-Host).UI.RawUI.BufferSize = New-Object Management.Automation.Host.Size (4096, 4096)
Get-Process

This produce text output with many empty spaces and diffplex returns 8000 tokens for each line of code (for each space it returns 2 tokens)

The fix is just optimizing SmartSplit method to be smarter and aggregate spacing characters

mmanela commented 5 years ago

Looks good, thanks for the contribution