search

FGasper / zmodemjs

zmodem.js - ZMODEM in JavaScript
Apache License 2.0
129 stars 25 forks source link
javascript shell terminal websocket xmodem ymodem zmodem

readme

zmodem.js - ZMODEM for JavaScript

build status

SYNOPSIS

let zsentry = new Zmodem.Sentry( {
    to_terminal(octets) { .. },  //i.e. send to the terminal

    sender(octets) { .. },  //i.e. send to the ZMODEM peer

    on_detect(detection) { .. },  //for when Sentry detects a new ZMODEM

    on_retract() { .. },  //for when Sentry retracts a Detection
} );

//We have to configure whatever gives us new input to send that
//input to zsentry.
//
//EXAMPLE: From web browsers that use WebSocket …
//
ws.addEventListener("message", function(evt) {
    zsentry.consume(evt.data);
} );

The on_detect(detection) function call is probably the most complex piece of the above; one potential implementation might look like:

on_detect(detection) {

    //Do this if we determine that what looked like a ZMODEM session
    //is actually not meant to be ZMODEM.
    if (no_good) {
        detection.deny();
        return;
    }

    zsession = detection.confirm();

    if (zsession.type === "send") {

        //Send a group of files, e.g., from an <input>’s “.files”.
        //There are events you can listen for here as well,
        //e.g., to update a progress meter.
        Zmodem.Browser.send_files( zsession, files_obj );
    }
    else {
        zsession.on("offer", (xfer) => {

            //Do this if you don’t want the offered file.
            if (no_good) {
                xfer.skip();
                return;
            }

            xfer.accept().then( () => {

                //Now you need some mechanism to save the file.
                //An example of how you can do this in a browser:
                Zmodem.Browser.save_to_disk(
                    xfer.get_payloads(),
                    xfer.get_details().name
                );
            } );
        });

        zsession.start();
    }
}

DESCRIPTION

zmodem.js is a JavaScript implementation of the ZMODEM file transfer protocol, which facilitates file transfers via a terminal.

STATUS

This library is BETA quality. It should be safe for general use, but breaking changes may still happen.

HOW TO USE THIS LIBRARY

The basic workflow is:

  1. Create a Zmodem.Sentry object. This object must scan all input for a ZMODEM initialization string. See zsentry.js’s documentation for more details.

  2. Once that initialization is found, the on_detect event is fired with a Detection object as parameter. At this point you can deny() that Detection or confirm() it; the latter will return a Session object.

  3. Now you do the actual file transfer(s):

    • If the session is a receive session, do something like this:

      zsession.on("offer", (offer) => { ... });
    let { name, size, mtime, mode, serial, files_remaining, bytes_remaining } = offer.get_details();

    offer.skip();

    //...or:

    offer.on("input", (octets) => { ... });

    //accept()’s return resolves when the transfer is complete.
    offer.accept().then(() => { ... });
});
zsession.on("session_end", () => { ... });
zsession.start();

      The offer handler receives an Offer object. This object exposes the details about the transfer offer. The object also exposes controls for skipping or accepting the offer.

    • Otherwise, your session is a send session. Now the user chooses zero or more files to send. For each of these you should do:

      zsession.send_offer( { ... } ).then( (xfer) => {
    if (!xfer) ... //skipped

    else {
        xfer.send( chunk );
        xfer.end( chunk ).then(after_end);
    }
} );

      Note that xfer.end()’s return is a Promise. The resolution of this Promise is the point at which either to send another offer or to do:

      zsession.close().then( () => { ... } );

      The close() Promise’s resolution is the point at which the session has ended successfully.

That should be all you need. If you want to go deeper, though, each module in this distribution has JSDoc and unit tests.

RATIONALE

ZMODEM facilitates terminal-based file transfers. This was an important capability in the 1980s and early 1990s because most modem use was for terminal applications, especially BBSes. (This was how, for example, popular shareware games like Wolfenstein 3D were often distributed.) The World Wide Web in the mid-1990s, however, proved a more convenient way to accomplish most of what BBSes were useful for, as a result of which the problem that ZMODEM solved became a much less important one.

ZMODEM stuck around, though, as it remained a convenient solution for terminal users who didn’t want open a separate session to transfer a file. Uwe Ohse’s lrzsz package provided a portable C implementation of the protocol (reworked from the last public domain release of the original code) that is installed on many systems today.

Where lrzsz can’t reach, though, is terminals that don’t have command-line access—such as terminals that run in JavaScript. Now that WebSocket makes real-time applications like terminals possible in a web browser, there is a use case for a JavaScript implementation of ZMODEM to allow file transfers in this context.

GENERAL FLOW OF A ZMODEM SESSION:

The following is an overview of an error-free ZMODEM session.

  1. If you call the sz command (or equivalent), that command will send a special ZRQINIT “pre-header” to signal your terminal to be a ZMODEM receiver.

  2. The receiver, upon recognizing the ZRQINIT header, responds with a ZRINIT header.

  3. The sender sends a ZFILE header along with information about the file. (This may also include the size and file count for the entire batch of files.)

  4. The recipient either accepts the file or skips it.

  5. If the recipient did not skip the file, then the sender sends the file contents. At the end the sender sends a ZEOF header to let the recipient know this file is done.

  6. The recipient sends another ZRINIT header. This lets the sender know that the recipient confirms receipt of the entire file.

  7. Repeat steps 2-5 until the sender has no more files to send.

  8. Once the sender has no more files to send, the sender sends a ZEOF header, which the recipient echoes back. The sender closes the session by sending OO (“over and out”).

PROTOCOL NOTES AND ASSUMPTIONS

Here are some notes about this particular implementation.

Particular notes:

IMPLEMENTATION NOTES

PROTOCOL CHOICE

Both XMODEM and YMODEM (including the latter’s many variants) require the receiver to initiate the session by sending a “magic character” (ASCII SOH); the problem is that there’s nothing in the protocol to prompt the receiver to do so. ZMODEM is sender-driven, so the terminal can show a notice that says, “Do you want to receive a file?”

This is a shame because these other two protocols are a good deal simpler than ZMODEM. The YMODEM-g variant in particular would be better-suited to our purpose because it doesn’t “litter” the transfer with CRCs.

There is also Kermit, which seems to be more standardized than ZMODEM but much more complex.

DESIGN NOTES

zmodem.js tries to avoid “useless” states: either we fail completely, or we succeed. To that end, some callbacks are required arguments (e.g., the Sentry constructor’s to_terminal argument), while others are registered separately.

Likewise, for this reason some of the session-level logic is exposed only through the Transfer and Offer objects. The Session creates these internally then exposes them via callback

SOURCES

ZMODEM is not standardized in a nice, clean, official RFC like DNS or HTTP; rather, it was one guy’s solution to a particular problem. There is documentation, but it’s not as helpful as it might be; for example, there’s only one example workflow given, and it’s a “happy-path” transmission of a single file.

As part of writing zmodem.js I’ve culled together various resources about the protocol. As far as I know these are the best sources for information on ZMODEM.

Two documents that describe ZMODEM are saved in the repository for reference. The first is the closest there is to an official ZMODEM specification: a description of the protocol from its author, Chuck Forsberg. The second seems to be based on the first and comes from Jacques Mattheij.

HISTORICAL: The included rzsz.zip file (fetched from ftp://archives.thebbs.org/file_transfer_protocols/ on 16 October 2017) is the last public domain release from Forsberg. http://freeware.nekochan.net/source/rzsz/ has what is supposedly Forsberg’s last shareware release; I have not looked at it except for the README. I’m not sure of the copyright status of this software: Forsberg is deceased, and his company appears to be defunct. Regardless, neither it nor its public domain predecessor is likely in widespread use.

Here are some other available ZMODEM implementations:

REQUIREMENTS

This library only supports modern browsers. There is no support for Internet Explorer or other older browsers planned.

The tests have run successfully against node.js version 8.

DOCUMENTATION

Besides this document, each module has inline jsdoc. You can see it by running yarn in the repository’s root directory; the documentation will build in a newly-created documentation directory.

CONTRIBUTING

Submit pull requests via GitHub.

TROUBLESHOOTING

Before you do anything else, set Zmodem.DEBUG to true. This will log useful information about the ZMODEM session to your JavaScript console. That may give you all you need to fix your problem.

If you have trouble transferring files, try these diagnostics:

  1. Transfer an empty file. (Run touch empty.bin to create one named empty.bin.)

  2. Transfer a small file. (echo hello > small.txt)

  3. Transfer a file that contains all possible octets. (perl -e 'print chr for 0 .. 255' > all_bytes.bin)

  4. If a specific file fails, does it still fail if you truncate a copy of the file down to, say, half size and transfer that truncated file? Does it work if you truncate the file down to 1 byte? If so, then use this method to determine which specific place in the file triggers the transfer error.

IF YOU HAVE DONE THE ABOVE and still think the problem is with zmodem.js, you can file a bug report. Note that, historically, most bug reports have reflected implementation errors rather than bugs in zmodem.js.

If you just want help using zmodem.js, try Gitter.

TODO

KNOWN ISSUES

USERS

The following are projects I’m aware of that use this library:

Send a pull request if you’d like your project listed.

COPYRIGHT

Copyright 2017 Gasper Software Consulting

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Parts of the CRC-16 logic are adapted from crc-js by Johannes Rudolph.