the encoding declaration in @types/crawler

[wubinhong]

Issue Description

In the tutorial of Crawler

I cannot assign a null value to CrawlerRequestOptions, when following the tutorial as below:

Trouble-shooting

encoding defined in @types/crawler

    interface CrawlerRequestOptions {
        encoding?: string | undefined;
    }

The implementation for encoding in file lib/crawler.js

Suggestion

Solution 1 (Recommended)

Change the encoding declaration in @types/crawler as below

    interface CrawlerRequestOptions {
        encoding?: string | undefined | null;
    }

Solution 2

Change the implementation of handling encoding as below:

Crawler.prototype._doEncoding = function (options, response) {
    var self = this;

    if (!options.encoding) {
        return;
    }

    if (options.forceUTF8) {
        var charset = options.incomingEncoding || self._parseCharset(response);
        response.charset = charset;
        log('debug', 'Charset ' + charset);

        if (charset !== 'utf-8' && charset !== 'ascii') {// convert response.body into 'utf-8' encoded buffer
            response.body = iconvLite.decode(response.body, charset);
        }
    }

    response.body = response.body.toString();
};

[mike442144]

Hi guy, I think this should be a bug of type inference in typescript. I don't like typescript, also suggest you no to use it.