search

silnrsi / smith

font development, testing and release
Other
14 stars 5 forks source link

smith xtest reports a difference between Graphite and OT rendering when the only difference is how they handle a BOM #15

Closed JBrase closed 7 years ago

JBrase commented 7 years ago

I recently divided a large test file into several smaller ones using Notepad, which inserted a BOM at the beginning of every file. Now "smith xtest" reports a difference between the graphite and OT rendering for every test file, even though many times the only difference is in how the BOM is handled.

image

mhosken commented 7 years ago

Then don't use BOMs :) or strip them or don't use Notepad.

bobh0303 commented 7 years ago

Personally, I'd prefer a discussion of why smith has to care about BOMs -- why doesn't it just ignore them like a good boy?

devosb commented 7 years ago

@JBrase and @bobh0303 might disagree with @mhosken's reasoning. From reading the TUS, it seems to me that a BOM on UTF-8 is not recommended, and while TUS mentions having such a BOM on text files as an aide to distinguish the encoding form, is that good practice in the environment (such as Linux) specifies UTF-8 in it's locale setting? TextEdit on macOS ignores a BOM, but gedit on Linux does not. So you have to be careful with the workflow.

Regardless of any code change or not, I think developer education is needed. If you are not careful, you can wind up with U+FEFF in the middle of the data (where it is not really a BOM). Plus, if you take a test file (with no BOM), edit the file in Notepad, the first line will show (I think) a diff in git since a BOM has been added to that first line. But the diff will be confusing, because the visible text will be the same.

Despite adding a BOM automatically, Notepad is useful for testing OT code on Windows, without the extra layers that Word adds. As far as I can tell, Notepad will open UTF-8 files without a BOM correctly, it is only when saving a file with Notepad does the BOM get added.

mhosken commented 7 years ago

It's not a smith issue as to BOM handling, but of the testing tool. So we could raise a bug against the test tool to ignore BOM, which wouldn't be hard. Anyway, I am going to close this bug again because it isn't a smith issue on 2 counts now :)