search

creationix / wheat

Wheat is a blog engine for coders written in node.JS
Other
1.36k stars 134 forks source link

Assume UTF-8 encoding for .markdown and .haml #30

Closed pirxpilot closed 13 years ago

pirxpilot commented 13 years ago

Alternative fix for #29 and #16

New Tools.fixEncoding function can be used to adjust the result of Git.readFile whenever we expect UTF-8 encoded file.

Should be pretty safe since ASCII is UFT-8 The real culprit here is git-fs which reads files to string using 'binary' encoding We convert the file content back to buffer and re-encode it as UTF-8

It would be better if git-fs were fixed to read files content into buffer (and not into string as it does now). But even after git-fs is fixed this code should work.

TooTallNate commented 13 years ago

Considering that the binary encoding has been deprecated for some time now, I think the real fix would be to make git-fs return Buffers, instead of binary Strings. Then UTF8 support would be as simple as buf.toString('utf8')...

pirxpilot commented 13 years ago

That's kind of what I was trying to say ;-) I don't know how many people/apps depend on Git.readFile returning strings. I'll try to prepare git-fs patch as well, but - as I said in the comment - even after git-fs is patched this code will still work, since the conversion to 'binary' only kicks in if Git.read file returns string.

pirxpilot commented 13 years ago

This is node-git pull request for fixing Git.fileRead

pirxpilot commented 13 years ago

Looks like @creationix pulled my git-fs patch. I am closing this one and will publish a patch based on new git-fs functionality.