node-downloader-helper

A simple http file downloader for node.js

Features:

• No thirdparty dependecies
• Pause/Resume
• Retry on fail
• Supports http/https
• Supports http redirects
• Supports pipes
• Custom native http request options
• Usable on vanilla nodejs, electron, nwjs
• Progress stats

Install

$ npm install --save node-downloader-helper

Usage

For a more complete example check example folder

const { DownloaderHelper } = require('node-downloader-helper');
const dl = new DownloaderHelper('https://proof.ovh.net/files/1Gb.dat', __dirname);

dl.on('end', () => console.log('Download Completed'));
dl.on('error', (err) => console.log('Download Failed', err));
dl.start().catch(err => console.error(err));

IMPORTANT NOTE: I highly recommend to use both .on('error') and .start().catch, although they do the same thing, if on('error') is not defined, an error will be thrown when the error event is emitted and not listing, this is because EventEmitter is designed to throw an unhandled error event error if not been listened and is too late to change it now.

CLI

This can be used as standalone CLI downloader

Install npm i -g node-downloader-helper

Usage: ndh [folder] [url]

$ ndh ./folder http://url

Options

Download Helper constructor also allow a 3rd parameter to set some options constructor(url, destinationFolder, options), these are the default values

{
    body: null, //  Request body, can be any, string, object, etc.
    method: 'GET', // Request Method Verb
    headers: {},  // Custom HTTP Header ex: Authorization, User-Agent
    timeout: -1, // Request timeout in milliseconds (-1 use default), is the equivalent of 'httpRequestOptions: { timeout: value }' (also applied to https)
    metadata: {}, // custom metadata for the user retrieve later (default:null)
    resumeOnIncomplete: true, // Resume download if the file is incomplete (set false if using any pipe that modifies the file)
    resumeOnIncompleteMaxRetry: 5, // Max retry when resumeOnIncomplete is true
    resumeIfFileExists: false, // it will resume if a file already exists and is not completed, you might want to set removeOnStop and removeOnFail to false. If you used pipe for compression it will produce corrupted files
    fileName: string|cb(fileName, filePath, contentType)|{name, ext}, // Custom filename when saved
    retry: false, // { maxRetries: number, delay: number in ms } or false to disable (default)
    forceResume: false, // If the server does not return the "accept-ranges" header, can be force if it does support it
    removeOnStop: true, // remove the file when is stopped (default:true)
    removeOnFail: true, // remove the file when fail (default:true)
    progressThrottle: 1000, // interval time of the 'progress.throttled' event will be emitted
    override: boolean|{skip, skipSmaller}, // Behavior when local file already exists
    httpRequestOptions: {}, // Override the http request options  
    httpsRequestOptions: {}, // Override the https request options, ex: to add SSL Certs
}

for body you can provide any parameter accepted by http.request write function req.write(body) https://nodejs.org/api/http.html, when using this, you might need to add the content-length and content-type header in addition with the http method POST or PUT

ex:

const data = JSON.stringify({
  todo: 'Buy the milk'
});
const dl = new DownloaderHelper('my_url', __dirname, { 
method: 'POST',
body: data,
headers: {
    'Content-Type': 'application/json',
    'Content-Length': data.length
} } );

for fileName you can provide 3 types of parameter

• string: will use the full string as the filename including extension
• callback(fileName, filePath, contentType): must return an string, only sync function are supported ex: (fileName) => 'PREFIX_' + fileName;, contentType will be provided if available
• object: this object must contain a name attribute and an optional ext attribute, the ext attribute can be an string without dot(.) or a boolean where true use the name as full file name (same as just giving an string to the fileName parameter) or false (default) will only replace the name and keep the original extension, for example if the original name is myfile.zip and the option is {name: 'somename'} the output will be somename.zip

for override you can provide 2 types of parameter

• boolean: true to override existing local file, false to append '(number)' to new file name
• object: object with properties skip (boolean): whether to skip download if file exists, and skipSmaller (boolean): whether to skip download if file exists but is smaller. Both default to false, for the equivalent of override: true.

for httpRequestOptions the available options are detailed in here https://nodejs.org/api/http.html#http_http_request_options_callback

for httpsRequestOptions the available options are detailed in here https://nodejs.org/api/https.html#https_https_request_options_callback

Methods

usage of resumeFromFile

const downloadDir = 'D:/TEMP';
const { DownloaderHelper } = require('node-downloader-helper');
const dl = new DownloaderHelper('https://proof.ovh.net/files/1Gb.dat', downloadDir);
dl.on('end', () => console.log('Download Completed'));
dl.on('error', (err) => console.log('Download Failed', err));

// option 1
const prevFilePath = `${downloadDir}/1Gb.dat`;
dl.resumeFromFile(prevFilePath).catch(err => console.error(err));

// option 2
const prevState = dl.getResumeState(); // this should be stored in a file, localStorage, db, etc in a previous process for example on 'stop'
dl.resumeFromFile(prevState.filePath, prevState).catch(err => console.error(err));

Events

event skip skipInfo object

{
    totalSize:, // total file size got from the server (will be set as 'null' if content-length header is not available)
    fileName:, // original file name
    filePath:, // original path name
    downloadedSize:, // the downloaded amount
}

event download downloadInfo object

{
    totalSize:, // total file size got from the server (will be set as 'null' if content-length header is not available)
    fileName:, // assigned name
    filePath:, // download path
    isResumed:, // if the download is a resume,
    downloadedSize:, // the downloaded amount (only if is resumed otherwise always 0)
}

event progress or progress.throttled stats object

{
    name:, // file name
    total:, // total size that needs to be downloaded in bytes, (will be set as 'null' if content-length header is not available)
    downloaded:, // downloaded size in bytes
    progress:, // progress porcentage 0-100%, (will be set as 0 if total is null)
    speed: // download speed in bytes
}

event end downloadInfo object

{
    fileName:, 
    filePath:,
    totalSize:, // total file size got from the server, (will be set as 'null' if content-length header is not available)
    incomplete:, // true/false if the download endend but still incomplete, set as 'false' if totalSize is null
    onDiskSize, // total size of file on the disk
    downloadedSize:, // the total size downloaded
}

event renamed filePaths object

{
    path:, // modified path name
    fileName:, // modified file name
    prevPath:, // original path name
    prevFileName:, // original file name
}

event error error object

{
    message:, // Error message
    status:, // Http status response if available
    body:, // Http body response if available
}

States

Test

$ npm test

License

Read License for more licensing information.

Contributing

Read here for more information.

TODO

• Better code testing