Consider migrating from resumable.js to something else

[fohrloop]

From https://github.com/23/resumable.js/issues/533 seems that resumable.js not maintained anymore. Flow.js if a fork of resumable.js, which has added functionality and seems to be actively maintained. The latest version of resumable.js on npm (1.1.0) is from Nov 2017, and latest version of flow.js on npm (2.14.1) is from June 2020.

There could be also some other interesting options.

I am open to suggestions, if anyone knows better than me what is out there.

[sorenwacker]

It sounds very reasonable to me. There is also https://www.dropzonejs.com/ and https://github.com/pqina/filepond. Personally, I am more interested in large batch compatibility than image previews and editability.

Though in general moving on from something that is not maintained anymore will be necessary at some point anyway.

Uppy also seems worthwhile looking at as described in this article. It can also get files from Dropbox/Google Drive, maybe a bit out of scope though.

[fohrloop]

I started a branch for v.0.5.0 which might contain some bigger changes. This is a candidate issue on it. So, if someone wants to experiment using some other library than resumable.js, better starting point (than master branch) would be https://github.com/np-8/dash-uploader/tree/0.5.0

[fohrloop]

Thanks for the tips, @soerendip 🙏

I had time to look a bit around. I also agree with you, and think that the top priorities for dash-uploader JS component would be

1. Ability to upload large files in small chunks. This is the selling point and reason dash-uploader exists.
2. Ability to upload many files at once. This is a feature request already on dash-uploader (https://github.com/np-8/dash-uploader/issues/20) and resumable.js seems to have a bug related to this. Preferably, user could just drag & drop a folder.
3. Robustness. The upload component must be robust for example when there are problems in internet connection.
4. Well documented API & easy to integrate with React
5. Fair amount of user / developer base & some recent commits

Points 1-3 are needed from the user perspective, points 4 is a developer-friendliness points and point 5 guarantees that the solution will last for some time.

Some thoughts on different options

• Dropzone.js: Does not seem to be possible to fulfill points 1 & 2 at once, as chunking option is not possible with uploadMultiple(?). Not sure but as it is not clear, not the first choice.
• filepond: Point 1 ok, "accepts directories" so I assume point 2 also ok (see issue 27 here). Nothing said about robustness (point 3). Seems well documented and has a react wrapper (point 4). Also point 5 ok. Seems very promising. Should check out the robustness, and not sure if resumable.
• Uppy: Point 1 ok (big files), there was demo with multiple files (question mark if folders can be selected), it is said to be robust and resumable (point 3), is well documented and has React components (point 4). Point 5 ok. Has also other goodies, like uploading directly to AWS S3. Definitely one candidate. (check: What does is mean "Uppy is not backend-free"?). Directory uploads seem not to be possible: issues 447 & 1027.
• Meteor-Files: Point 1 looks ok (chunks), multi-file uploads possible, but are folder uploads?, is robust (3), points 4 and 5 ok.
• Uppload: I could not find absolutely anything about chunks from the website. Could be that point 1 not fulfilled, as "Uppload is a better JavaScript image uploader." --> Not considering now.
• tus-js-client: Points 1 & 3 clearly met. Question remains on point 2: is uploading a folder structure possible? More about tus protocol here.
• react-dropzone: This is what the dcc.Upload component uses -> no.
• flow.js: Originally fork of resumable.js. Points 1 & 3 ok. Also 4 & 5. Not sure about uploading folders. This page says uploading a folder is possible, but here is this issue. -> Should Try
• Simple-Ajax-Uploader: Interesting but no recent activity in the project (point 5).
• Plupload: Point 1 ok, uploading folders possible (point 2). Resumability is a question mark. See their issue 57.
• Dropit: No references to points 1-3 --> dropping it.
• blueimp/jQuery-File-Upload: Seems to fulfill the points, but has jQuery dependency (dash-uploader JS components are created using React) -> Not considering first.

Summary

At first glance, looks like filepond and flow.js are strong candidates here. Perhaps also Plupload. These should be tested for (1) the ability to upload directory structures and (2) how they handle interrupts in the internet connection.

Am I missing something else here?

[fohrloop]

If anyone has expertise & time to check if any of the candidates (flow.js, filepond, Plupload, Meteor-Files, or others) meet the requirements -especially, uploading a folder full of files, it would be a great help. After that dash-uploader could start migration process.

[fohrloop]

Started a branch for flow.js. One of the first things should be to check if there is issue when uploading more than 100 files, as mentioned in github.com/flowjs/flow.js/issues #89.

[sorenwacker]

Seems all essential components are working in the FlowJS branch of my pull request. The last thing to add would be automatic tests. But, I am not sure how to do that. The test in the repo is actually not testing anything yet. But manual testing worked.

• upload multiple files
• resumable
• cancel button works
• continuous progress bar for multiple files

That was a great learning experience.

[sorenwacker]

Also, after the tests are added, we could clean up the code? There are still left overs from the resumable.js in there.

[sorenwacker]

The code was updated to the latest Dash version. So, it will not be compatible with older versions due to the different imports. E.g.

from dash import html

[fohrloop]

I think this issue can be closed as the consideration is done. Next step is to finish the flow.js implementation in the flow-dev branch.