iftechfoundation / ifarchive-unbox

IF Archive Unboxing service
https://unbox.ifarchive.org
MIT License
1 stars 2 forks source link

Don't use exec() for file #42

Closed erkyrath closed 2 years ago

erkyrath commented 2 years ago

Note: This PR is built on top of https://github.com/iftechfoundation/ifarchive-unbox/pull/41 ! Merge that first.

I've written a little utility to pipe the output of unzip/untar into file.

This uses the Node event-driven file-stream API. All the output of unzip -p xxx.zip file goes through the unbox process. But we don't store it all, not even in memory. It's just passed along chunk by chunk.

(The stdout/stderr output of file does get accumulated in memory. But that's tiny.)

Now, the bad news: this doesn't fix the problems that I originally thought it would fix. In https://github.com/iftechfoundation/ifarchive-unbox/issues/39 , I mentioned two files causing errors:

Turns out these are not shell bugs. They are mismatches between what unzip reports in its file list and what it accepts as a filename. Oh well!

(I recognize the Icon^M thing. It's a MacOS Classic convention where a folder icon appeared in the filesystem with that filename, yes, with a ctrl-M in it. Everybody hated that.)

Anyhow, this spurred me into cleaning up the exec() situation, and I'm very happy about that. I think my implementation is correct. It's possible I missed something; I still haven't spent a lot of time in async-node-land.

erkyrath commented 2 years ago

I see the lint error but I don't know what the idiom is to make it go away.

curiousdannii commented 2 years ago

Arrow functions with no arguments have the syntax of () => {}. Or function() {} works too ;).

erkyrath commented 2 years ago

Thanks -- I knew it had to be something obvious.