node-js-libs / curlrequest

A cURL wrapper
MIT License
184 stars 44 forks source link

No data returned when remote file exceeds certain size #54

Open renefournier opened 7 years ago

renefournier commented 7 years ago

Also, this is intermittent, sometimes it works, usually does not:

` 'use strict';

const curl = require('curlrequest') ;

const feed = 'http://theartofcharmpodcast.theartofcharm.libsynpro.com/rss'; # Around 2.5MB

let options = { url: feed, verbose: true, stderr: true, retries: 3, timeout: true, headers: { 'Accept-Encoding': 'gzip' } }; curl.request(options, (err, data) => { if (err) { console.error (: C ${feed} [ERROR] ${err}); } else { if ( ! data) { console.log (: B ${feed} [NO DATA]); } else { console.log (: B ${feed} ${data.length}); } } });

`

renefournier commented 7 years ago

I realized it's because the timeout is set to true, and the request fails because the download exceeds the default limit. Problem is, when I specify timeout: 30, I get an error "[ERROR] Failed to connect to host" every time.