Closed matthoskins1980 closed 8 years ago
Well, there are different reasons why these extensions haven't been implemented yet:
To summaries, there is no real reason why extension support is not high right now, except the (maybe) missing needs and technical details. A user could build the required code on top of tus-js-client but is not supposed to do so. I hope to improve the situation soon.
@Acconut makes total sense - and please don't take my question as criticism - this protocol definition is exactly what we (the webdev community) needed!
I want to make sure I understand the Concatenation extension - let me explain my use case and see if it fits. I want to give my users the ability to upload medium-to-large video files (in excess of 350MB) over a mobile hotspot. The browser will be on a laptop (not mobile). In order to maximize network performance - I like to make the upload multi-threaded. Essentially - I'd like to upload one file over 3 to 5 HTTP connections. Could concatenation be used for this?
Yes, that's one of the use cases we designed the Concatenation extension for.
@Acconut I like your explanations in https://github.com/tus/tus-js-client/issues/29#issuecomment-171659656, would it make sense to cement (some of) those in README.md
?
FWIW, here is a function I came up with for generating an MD5 hash of a File
object using spark-md5 and chunked-file-reader:
function fileMD5 (file) {
return new Promise(function (resolve, reject) {
var spark = new SparkMD5.ArrayBuffer(),
reader = new ChunkedFileReader();
reader.subscribe('chunk', function (e) {
spark.append(e.chunk);
});
reader.subscribe('end', function (e) {
var rawHash = spark.end(true);
var base64Hash = btoa(rawHash);
resolve(base64Hash);
});
reader.readChunks(file);
})
}
The chunked-file-reader
is probably not going to be necessary, since tus-js-client is splitting the File
object into chunks of Blob
objects which are already loaded whole into memory.
For files of 20MB this takes about 600ms on my machine. I don't expect anyone to use chunks larger than 20MB, so I think it's a pretty small overhead. I haven't compared spark-md5 with other alternatives, but spark-md5 claims to be very fast.
@janko-m Those are some promising numbers, thanks for sharing!
Another update, the default chunk size for ChunkedFileReader
is 256 KB, which I think is bit too small. I tried bumping the chunk size to 2MB, which I think is very reasonable, and for me that sped up hashing from 600ms to 200ms (for a 20MB file).
So, I think we should be good regarding the performance of calculating a checksum, though it would probably be good to also test the speed on mobile devices.
From what I can tell, the tus-js-client does not support many of the extensions in the tus protocol - namely - checksum and concatenation. Are those extensions left to the user to implement outside of the client library? Or is it just a matter of time / needing implementation? Or - am I just missing something completely? :)