search

nacholibre / node-readlines

Read large text files line by line (synchronous) for Node.js
MIT License
162 stars 28 forks source link

UTF-16 #21

Open netmiller opened 6 years ago

netmiller commented 6 years ago

Any advise how to read utf-16 (LittleEndian) file info buffer. Tried normal way :

while (row = reader.next()) {
    line = row.toString('utf16le');
    ........
}

but seems to be incorrect ?

tracker1 commented 5 years ago

would probably be best to fork and update the search for the newline pairing. just curious why not utf8?

jforge commented 3 years ago

would probably be best to fork and update the search for the newline pairing. just curious why not utf8?

The data source could be UTF-16LE encoded, and one wants to handle raw lines of data. Example: sync, read/compare windows files from a machine with fixed encoding (UTF-16LE)

Actually the project should support any source encoding.

quoininc-huudatran commented 3 years ago

Actually the project should support any source encoding.

I'm not sure if this is still an issue, but I believe it's working? I have a dynamic encoding detection with chardet, and I use that value with row.toString(encoding) and I correctly get the string I want.. from my code samples.

@netmiller The exact encoding I use is UTF-16LE. Maybe you need to pass that instead of utf16le.