search

jeanphix / django-resumable

django backend for resumable.js xhr uploads.
61 stars 21 forks source link

More efficient chunk merging #3

Closed EnTeQuAk closed 9 years ago

EnTeQuAk commented 11 years ago

NOTE: This requires https://github.com/23/resumable.js/pull/36 as I introduced a size checking for each chunk to harden the server against network failures or browser reloads.

NOTE 2: I am working on adapting the unittests properly so this PR is basically to get early feedback from you so that I can probably adapt to other use cases as well.

Implementation:

We must not read the file contents into memory, especially if we deal with large amount of data which is the main use case of chunked uploads.

This commit fixes the old behavior and introduces a more efficient merging. We still read every chunk into memory but this happens in a generator and we do not merge in-memory so that the memory usage stays constant.

EnTeQuAk commented 11 years ago

Rebased to use https://github.com/23/resumable.js/pull/36

jeanphix commented 11 years ago

I that ready to merge? I have no time today to test it. Let me know.

Thanks

EnTeQuAk commented 11 years ago

No, it's not since it relies on another pull request on 23/resumable.js, except we distribute the library directly via the package instead of linking to the github page?

jeanphix commented 11 years ago

No objection. (One day I was thinking of rewriting a more clean resumable client library, don't realy like the code style and that there are no tests...)

EnTeQuAk commented 9 years ago

@jeanphix this is a amazingly old PR but since the required code already landed in resumable.js it should be save to merge this :smile:

jeanphix commented 9 years ago

@EnTeQuAk Ok, lets merge it then :+1: