Support non-UTF8 encodings - Githubissues

atom / scandal

Efficient directory scan + search utilities

http://atom.github.io/scandal

MIT License

47 stars 29 forks source link

Support non-UTF8 encodings #26

Open benogle opened 9 years ago

benogle commented 9 years ago

Thanks to @aki77 we are better at handling utf8 with #25. But support for any other encodings is terrible.

[ ] It assumes many non-utf-8 files are binary. Fix / get https://github.com/gjtorikian/isBinaryFile/issues/22 fixed
[ ] Node only supports a few encodings. No bueno. We should use something like https://github.com/ashtuchkin/iconv-lite (which is what pathwatcher uses.
[ ] Expose the encoding option to PathSearcher and PathReplacer

Potential files for testing: https://github.com/benogle/encodings

Open questions

With open files, we can pass in the encoding from Atom. How to effectively search files with different encodings? I suppose we need to solve the encoding autodetection in core, then we can use the same pattern when opening each file.
Same goes for replacing. We need to write the file in the correct encoding.