hgouveia / node-downloader-helper

A simple http file downloader for node.js
MIT License
253 stars 54 forks source link

resumeFromFile not working through proxy #91

Closed khkassem closed 1 year ago

khkassem commented 1 year ago

Hello

I am trying to use the resumeFromFile function through a proxy with authentication and it seems not working. when I use the same code without proxy it's downloading fine.

      newDownloadHelper.on('start', startFunction)
      newDownloadHelper.on('download', startFunction)
      if (fs.existsSync(prevFilePath)) {
        logger.silly(`download manager before calling resumeFromfile ${prevFilePath}`)
        newDownloadHelper.resumeFromFile(prevFilePath)
        logger.silly(`download manager after calling resumeFromfile ${prevFilePath}`)
      } else {
        newDownloadHelper.start()
      }

The events start and download are not called when using a proxy, if the file (prevFilePath) is not found, I call start() all is OK with or without proxy.

khkassem commented 1 year ago

Hello

I retried from my side, on a SQUID Proxy with basic authentication and it is working, but on more than 50 users I get no download start when calling resumeFromFile, even if the file exists... No event start or download fired, so I don't know if its something on the proxy, or i missed something which squid proxy can manage it but not the users proxy (I don't know what type of proxy they use)

hgouveia commented 1 year ago

@khkassem i believe is a bug on my part i did check it as soon i got the issue, but the real issue is in the "getTotalSize" https://github.com/hgouveia/node-downloader-helper/blob/master/src/index.js#L391 that is not reusing the http/httpsRequestOptions so is not using the Agent with proxy https://github.com/hgouveia/node-downloader-helper/blob/master/src/index.js#L357, i will fix that when i have the time

in theory you can workaround if you send the filename and total file size to resumeFromFile, since it will bypass the getTotalSize function

const totalSize = await myCustomFuncToGetFileSize(url);
dl.resumeFromFile(filePath, { name: 'myFileName', total: totalSize });
khkassem commented 1 year ago

Hello

If it may help, i override the DownloadHelpper class and I changed the __getOptions function adding this line

httpsRequestOptions: this.getOptions()['httpsRequestOptions'],

export class EzDownloaderHelper extends DownloaderHelper {
  server_id: number = -1

  constructor(server_id, url: string, destFolder: string, options?: DownloaderHelperOptions) {
    super(url, destFolder, options)
    this.server_id = server_id
  }

  __getOptions(method, url, headers = {}) {
    const urlParse = new URL(url)
    const options = {
      protocol: urlParse.protocol,
      host: urlParse.hostname,
      port: urlParse.port,
      path: urlParse.pathname + urlParse.search,
      method,
      httpsRequestOptions: this.getOptions()['httpsRequestOptions'],
      httpRequestOptions: {}, // Override the http request options
    }

    if (headers) {
      options['headers'] = headers
    }

    return options
  }

}

and it seems OK, i am going to test it on another proxy configuration

hgouveia commented 1 year ago

@khkassem this should be fixed now in v2.1.5, let me know if this solved it for you