search

adamreisnz / replace-in-file

A simple utility to quickly replace contents in one or more files
580 stars 65 forks source link

Add support for streaming large files #128

Open mykola-mokhnach opened 4 years ago

mykola-mokhnach commented 4 years ago

Hello Adam,

Thanks for the great library, it's really useful. I'd like to mention one thing about its usage: when trying to perform replacements in big files (like hundreds of megabytes) then OutOfMemory errors could easily pop up, because the lib loads the file content fully into the memory before, actually, making the replacement. I'm not quite sure whether this was done by design, but it, probably, makes sense to at least mention this fact in README.

An alternative solution, which is memory-friendly (uses streams), could be something like https://github.com/eugeneware/replacestream

adamreisnz commented 4 years ago

Hi there, yep some work on this was started in https://github.com/adamreisnz/replace-in-file/issues/40, but that has eventually been abandoned.

I'd be happy to review and accept PR's that implement some kind of streaming option or support for large files.

However, I'm not sure how the replace algorithm will work correctly if the file contents are read in chunks (for example, the regex may not find a match when a chunk is split exactly at a match).

Noeldesarrollo commented 3 years ago

Hello @adamreisnz I have a solution implemented and tested, but i don't have permissions to create a branch.

adamreisnz commented 3 years ago

Hi @Noeldesarrollo , please fork the repository and then open a pull request, and I'll be able to have a look. Thanks!

Noeldesarrollo commented 3 years ago

PR: https://github.com/adamreisnz/replace-in-file/pull/139