szwacz / fs-jetpack

Better file system API for Node.js
MIT License
777 stars 41 forks source link

Allow passing encoding option to write and read #75

Closed matortheeternal closed 4 years ago

matortheeternal commented 6 years ago

I have a program where I need to read and write some ucs2 encoded text files. fs-jetpack doesn't have a way to specify encoding in the read and write methods, so I have to switch to using fs directly for these operations. To avoid this, it would be really cool if fs-jetpack allowed specifying an encoding option when using write and read that would get passed to the associated fs function when called.

Again, happy to make a PR if this change is desirable and within the intended scope of this project.

szwacz commented 6 years ago

Definitely! But fs.writeFile() supports any of the encodings Buffer is supporting: https://nodejs.org/api/buffer.html#buffer_buffers_and_character_encodings. So the list is rather long, let me think how to tackle this.

szwacz commented 6 years ago

Also this feature is actually a sugar syntax over:

jetpack.read("foo.txt", "buffer").toString("ucs2")
papb commented 5 years ago

Hello @matortheeternal and @szwacz, I would also like to see an option to choose the encoding. I need to write a file in 'binary' encoding, but sadly jetpack.write() only allows 'utf8' encoding (which is the default of fs).

I am using fs.writeFileSync('fileName', data, 'binary') for now but I would really like to use fs-jetpack here.

szwacz commented 4 years ago

I’ve looked into it again and for me it makes more sense to leave the api as is right now. First of all, achieving all encodings is already possible, because those are supported by buffer api, not by file system api, so what you need to do is:

// To read any encoding
const myData = jetpack.read("foo.txt", "buffer").toString("utf16le")

// To write any encoding
jetpack.write("foo.txt", Buffer.from(myData, "utf16le")

Node’s file system api allows to specify any encoding directly in the method calls as a sugar syntax, but for me it’s obscuring what actually is going on (their own docs aren’t actually clear about what encodings you can use). So in this case using Buffer as a middle man is better code pattern in my opinion.

The only possible pro I can see is if node's fs library is actually performing some memory optimization in the process so does reading file -> string instead of file -> buffer -> string. But I don't know if this is the case, and don't know how I can learn about this :)