Standalone CLI implementation?

[ianfixes]

For the purposes of CI/CD, it would be helpful to have a command-line tool that could encode and decode text via piping, similar to base64.

echo PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0i\
     VVRGLTgiPz4KPHRlc3RzdWl0ZSDFCj0iMyIgZmFp\
     bHVyZXM9IjEiIHRpbWU9IjAuMDE2MDEwNiI+CiAg\
     ICDFOGNhc2UgbmHEIGhhcyBodG1sIiBjbGFzc8YV\
     SFRNTMtBMDA3Ncc/xUNlcnJvciBtZXNzYWdlPSI8\
     Yj5CT088L2I+IiB0eXDEEmk+QVJHSDwvaT7LOCZs\
     dDtpJmd0O0FSR0fEDS/FDsQKYsQJQk9PxRbFDSA8\
     L8Vnxj88L+gAtz4KxgzlAPs+\
| base64 --decode \
| lzutf8 --decode           # which doesn't currently exist

How would I create such a tool?

[rotemdan]

There is actually an unofficial cli tool written in JavaScript / Node at cli/lzutf8-cli.js (It was written 6-7 years ago and hasn't been maintained since).

Based on the source, the tool relies on the library built in ../build/development/lzutf8.js. Running grunt in the base dir should work to build the library (the path seems to be valid).

Usage: node lzutf8-cli [command] [source] [destination?]
Commands:
  c   Compress [source] to [destination]
  d   Decompress [source] to [destination]
  t   Test compression and decompression correctness using [source]

I have tested the current version and it sees like it needed some additional process.exit() calls to work correctly with modern versions of Node. I've fixed the issue and committed it to version 0.6.0 of the library (now on GitHub).

[Making a proper, optimized tool would require rewriting the entire codebase in C/C++ as well as compiling the core to WebAssembly. The performance would be several times higher than today but it isn't something that I foresee to happen anytime soon (possibly never).]

[ianfixes]

I'm not worried about the performance, the major need is just to have a working implementation of the algorithm available on the command line.

Is this lzutf8-cli tool able to handle piping or does it just work on source/destination paths?

[ianfixes]

Also, thank you for moving so quickly on this!

[rotemdan]

@ianfixes The tool was made for my personal testing purposes.

It doesn't currently support reading from stdin / writing to stdout .

It isn't technically difficult to add support for it, but that would require changing the command line parameters. Essentially forking it and making a new tool.

The source code is here.

Both the "compress" and "decompress" operations have two streams as input / output

let sourceReadStream = NodeFS.createReadStream(sourceFilePath);
let destWriteStream = NodeFS.createWriteStream(destinationFilePath);

It is possible to change them to something like:

let sourceReadStream = process.stdin;
let destWriteStream = process.stdout;

Since this are standard streams, I'm not sure if that would work with the "close" event though:

resultStream.on("close", () => {
  ...
})

To get mixed piping and file i/o (say stdin as input and a file as output) would require more work.