How to download multiple files at once?

[abmagil]

I'm trying to download all the files from a gist and I'm having inconsistent behavior. A basic repro case can be found at this repo. My desired behavior would be that I could set a directory for all the files in a gist, but could change the directory on a gist-by-gist basis (so that directory would change multiple times within the app's lifetime).

The failed behavior manifests as repeated "Save as..." prompts, despite setting the directory option. If I cancel out of the prompts, my files do not download. Other times, the app will not prompt me, but will download one file (e.g. thumbnail.png) N times, where N is the number of files in the gist.

I suspect that this line, which calls download anew each iteration through an array is the root of my problem, but it's unclear to me what the intended call pattern is for my use case.

[ribsies]

Running into this issue as well.

It looks like the options are being set globally and it looks like only one download can exist at a time.

Lets say I have 10 files I am downloading, and calling the download function in a foreach function. By the end of it I only have 1 file downloaded, which has the contents of the first download in the list but the name of the last item.

I haven't had time to test but it looks like you might have to wait for a download to completely finish before doing another one. Which is not ideal.

[itzsaga]

We do it with this code for an array of objects where there is a url key on each object.

ipcMain.on("download-files", async (event, files) => {
  const downloadLocation = app.getPath("userData");

  const promises = files.map(file =>
    download(mainWindow, file.url, {
      saveAs: false,
      directory: downloadLocation,
      onProgress: progress => {
        event.sender.send("download-progress", { progress, file });
      }
    })
  );

  await Promise.all(promises);
});

[alexhx5]

@itzsaga @sindresorhus guys, do you have any advice on how to fix onProgress events with simultaneous downloading?

When I try to download 2nd file while 1st file is downloading, it downloads both files but it starts emitting onProgress events unexpectedly during the downloading.

Code

I modified the module a bit so it accepts other arguments as well, not just the url:

ipcMain.on('download-file', async (event, { url, directory, hash }) => {
  await download(mainWindow, { 
    url, 
    directory,
    hash,
    onProgress(progress) {
      event.sender.send('download-file-progress', progress)
    }
  })
})

Log:

• Started downloading 1st file blender-2.81.tar.xz
• After 5 seconds started downloading 2nd file blender-2.81-windows64.msi

I passed hash-1 for the 1st file and hash-2 for the 2nd into the download() with the file URL just for the log.

Compare the percentages and the hashes, look at how hashses do not correspond correctly to their files after I started downloading 2nd file:

blender-2.81.tar.xz  - 0.00% - hash-1
blender-2.81.tar.xz  - 1.88% - hash-1
blender-2.81.tar.xz  - 3.88% - hash-1
blender-2.81.tar.xz  - 5.89% - hash-1
blender-2.81.tar.xz  - 8.14% - hash-1
blender-2.81.tar.xz  - 10.40% - hash-1
blender-2.81.tar.xz  - 11.40% - hash-1
blender-2.81.tar.xz  - 13.65% - hash-1
blender-2.81.tar.xz  - 15.74% - hash-1
blender-2.81.tar.xz  - 17.79% - hash-1
blender-2.81-windows64.msi  - 0.00% - hash-1
blender-2.81-windows64.msi  - 0.00% - hash-2
blender-2.81-windows64.msi  - 0.00% - hash-1
blender-2.81-windows64.msi  - 0.00% - hash-2
blender-2.81-windows64.msi  - 19.83% - hash-1
blender-2.81-windows64.msi  - 0.50% - hash-1
blender-2.81-windows64.msi  - 0.50% - hash-2
blender-2.81-windows64.msi  - 21.92% - hash-1
blender-2.81-windows64.msi  - 1.56% - hash-1
blender-2.81-windows64.msi  - 1.56% - hash-2
blender-2.81-windows64.msi  - 23.21% - hash-1
blender-2.81-windows64.msi  - 2.77% - hash-1
blender-2.81-windows64.msi  - 2.77% - hash-2
blender-2.81-windows64.msi  - 24.51% - hash-1
blender-2.81-windows64.msi  - 4.03% - hash-1
blender-2.81-windows64.msi  - 4.03% - hash-2
blender-2.81-windows64.msi  - 26.01% - hash-1
blender-2.81-windows64.msi  - 5.33% - hash-1
blender-2.81-windows64.msi  - 5.33% - hash-2
blender-2.81-windows64.msi  - 27.30% - hash-1
blender-2.81-windows64.msi  - 6.48% - hash-1
blender-2.81-windows64.msi  - 6.48% - hash-2
blender-2.81-windows64.msi  - 28.64% - hash-1
...

[SushyDev]

having similar issues, downloading multiple files at the same time makes data leak into the wrong item. For example the progress of file 2 switches back and forth on file 1. So if file 1 is on 10% and file 2 on 20%, file 1 will show 10% then 20% then 10% then 20% on and on again

[kbuffington]

I spent a fair amount of time looking into this tonight and it seems that there's just no good way to handle simultaneous downloads. registerListener is going to create multiple listeners for 'will-download' and then when the downloads complete multiple listeners will be getting the callback and it becomes a whole mess, and all downloads get saved under the same file name, progress gets screwed up etc. Maybe there's a solution, but I couldn't figure one out.

What I did was make it so you can't have multiple simultaneous downloads by implementing a download queue:

class DownloadQueue extends Array {
    constructor(...args) {
        super(...args);
    }
    push(item) {
        const len = super.push(item);
        if (this.length === 1) {
            this.download(item);
        }
        return len;
    }
    shift() {
        const item = super.shift();
        if (this.length > 0) {
            this.download(this[0]);
        }
        return item;
    }
    download(item) {
        item.options.onCompleted = () => {
            this.shift();
        };
        download(item.win, item.url, item.options);
    }
}

And then your 'download' listener becomes:

const downloadQueue = new DownloadQueue();

ipcMain.on('download', async (event, info) => {
    info.win = BrowserWindow.getFocusedWindow();
    downloadQueue.push(info);
}

[lohannon]

We ran into this as well, where we needed to download two sets of files, each with a dynamic number of files. Some files from one set were getting written to the directory of the other. The way we had to get around this was to make everything synchronous. This ended up working for us:

async function downloadFile({ directory, url, filename }): Promise<string> {
  return new Promise((resolve) => {
    download(mainWindow, url, {
      directory,
      filename,
      saveAs: false,
      showBadge: false,
      onCompleted: ({ path }) => {
        resolve(path);
      },
    });
  });
}

async function downloadFiles(
  directory: string,
  files: Array<{ url: string; filename: string }>
): Promise<string[]> {
  return await files.reduce(
    async (result: Promise<Array<string>>, { url, filename }) => {
      const awaitedResults = await result;
      try {
        const downloadedFile = await downloadFile({ directory, url, filename });
        return [...awaitedResults, downloadedFile];
      } catch (err) {}
    },
    Promise.resolve([])
  );
}

[gihanrangana]

class DownloadManager {
    public window: any = null
    public paths: any[] = []
    public options: any = {}

    constructor (window: any, paths: any[], options: {} = {}) {
        this.window = window;
        this.paths = paths;
        this.options = options
    }

    download = async (options?: any) => {
        for (let item of this.paths) {
            const dl: any = await download(this.window, item.url, {
                ...this.options,
                filename: item.name + '.zip',
                onProgress: (progress: any) => {
                    App.window.webContents.send(options?.progressEventName || 'download-progress', { ...progress, key: item.name })
                }
            })
            if (dl.isDone) {
                App.window.webContents.send(options?.successEventName || 'download-complete', { key: item.name })
            }
        }
    }

}

[j-ribs]

@gihanrangana

Can you give some more context to your code? How and where are you calling this from? Whats the download function thats being awaited there. Missing some key points here.

[theogravity]

I wrote a library that addresses the multi-download issue and also supplies an id to the download as well:

https://www.npmjs.com/package/electron-dl-manager

[Bugrabugra]

I wrote a library that addresses the multi-download issue and also supplies an id to the download as well:

https://www.npmjs.com/package/electron-dl-manager

I have migrated my code to your library, thank you so much!