We saw an error when an ASCII-8BIT yarn.lock file was run against bibliothecary after adding the "remove BOM" logic:
Encoding::CompatibilityError: incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string)
We need to force the string's encoding to be utf8 before regex-ing it. ASCII-8BIT is fine to force, but if we need to support less common encodings like utf-32be in the future, we'll probably need to call encode() on the string too so it's properly encoded.
Followup to https://github.com/librariesio/bibliothecary/pull/564
We saw an error when an ASCII-8BIT yarn.lock file was run against bibliothecary after adding the "remove BOM" logic:
We need to force the string's encoding to be utf8 before regex-ing it. ASCII-8BIT is fine to force, but if we need to support less common encodings like
utf-32bein the future, we'll probably need to callencode()on the string too so it's properly encoded.