Fix error when buffer contains only part of 3byte UTF-8 characters.

coinbase / terraform-landscape

Improve Terraform's plan output to be easier to read and understand

Apache License 2.0

1.59k stars 116 forks source link

Fix error when buffer contains only part of 3byte UTF-8 characters. #65

Closed synthdnb closed 6 years ago

synthdnb commented 6 years ago

Currently landscape read output in unit of 1024bytes. When buffer contains only part of 3-byte UTF-8 character, this causes ArgumentError when matching with regex.

For example, 갱 is 3byte UTF-8 character "\xEA\xB0\xB1" and valid, but "\xEA\xB0" is invalid UTF8 string.

This PR fixes this problem by replacing invalid UTF8 character to empty string.

sds commented 6 years ago

Thanks for the quick fix, @synthdnb. Might have made sense to simply try and buffer another KB of data rather than lose the codepoint with an empty string, but this will suffice.