Chunk missing in upload

[bartbergmans]

I am using laravel-chuck-upload in combination with dropzone.js. Everything works fine, the chunks are uploading and when they are uploaded the final file will be saved in S3. The problem is that 1 chunk is always missing. Sometimes .8.part is missing, sometimes .7.part is missing (they remain in the chunks directory after uploading). When I upload a video that has a total size of 9.7MB the file in S3 is 8.7MB. That is 1mb missing, the same size as the missing part. All the chunks are 1MB in size.

What could be the problem and how can I fix this?

[bartbergmans]

I think I have found the problem, but not the fix. When the last chunk (10th) is uploaded it thinks all the chunks are uploaded but the 8th chunk isn't finished uploading yet.

[bartbergmans]

Extended the DropZoneUploadHandler with my own isLastChunk method. This way the file gets uploaded when it's really the latest chunk. The only problem is that dropzone.js triggers the chunksUploaded event for the last chunk call and not for the last completed chunk.

public function isLastChunk()
{
    $chunkFileName = preg_replace(
        "/\.[\d]+\.".ChunkStorage::CHUNK_EXTENSION.'$/', '', $this->getChunkFileName()
    );

    $files = ChunkStorage::storage()->files(function ($file) use ($chunkFileName) {
        return false === Str::contains($file, $chunkFileName);
    });

    return (count($files) + 1) == $this->getTotalChunks();
}

[bartbergmans]

Added a chunkUploaded function to dropzone.js to get the response of every chunk upload. Now I can do everything as I did before but with file that has all the chunks.

I added the following to dropzone.js:

/**
 * The callback that will be invoked when a single chunk has been uploaded for a file.
 * It gets the file for which the chunks have been uploaded as the first parameter,
 * and the `done` function as second. `done()` needs to be invoked when everything
 * needed to finish the upload process is done.
 */
chunkUploaded: function chunkUploaded(file, response, statuscode, done) {
    done();
},

xhr.onreadystatechange = function () {
    if(this.readyState == 4) {
        _this16.options.chunkUploaded(file, this.responseText, this.status, function () {
            _this16._finished(files, '', null);
        });
    }
};

[bartbergmans]

I have noticed that sometimes this isn't working. I have added the following to ChunkSave.php to make sure the isLastChunk is always set.

public function checkLastChunk()
    {
    $this->isLastChunk = $this->handler()->isLastChunk();

    return $this;
}

$this->handleChunkFile($file)
    ->checkLastChunk()
    ->tryToBuildFullFileFromChunks();

[LadyNay]

I am currently having this with my system as well. It just started all of a sudden. It works fine sometimes and then it leaves a chunk or a chunk is not merged. Is this what is happening with you?

[LadyNay]

Though I am not using the dropzone library, just the standard upload. This may prove to be a problem with this plugin

[LadyNay]

Did open ticket here, before seeing your post

https://github.com/23/resumable.js/issues/559

[kfeng0806]

You can check my answer here at @LadyDeathKZN 's issue https://github.com/23/resumable.js/issues/559#issuecomment-622429803

[MaximMonin]

I had the same problem with dropzone.js and laravel 7. If parallelChunkUploads: false everything seems work ok If I use parallelChunkUploads: true, It seems some chunks are loaded after last chunk.

[MaximMonin]

Extended the DropZoneUploadHandler with my own isLastChunk method. This way the file gets uploaded when it's really the latest chunk. The only problem is that dropzone.js triggers the chunksUploaded event for the last chunk call and not for the last completed chunk.

public function isLastChunk()
{
    $chunkFileName = preg_replace(
        "/\.[\d]+\.".ChunkStorage::CHUNK_EXTENSION.'$/', '', $this->getChunkFileName()
    );

    $files = ChunkStorage::storage()->files(function ($file) use ($chunkFileName) {
        return false === Str::contains($file, $chunkFileName);
    });

    return (count($files) + 1) == $this->getTotalChunks();
}

I tried to solve problem with method proposed and added PullRequest. Also checking uploaded size vs TotalFileSize to upload in ParallelChunksUpload Mode = true

[pionl]

Hi all, thanks @MaximMonin for a support in my absence.

Fixed a bug and added a support for Laravel 7 in v1.4.0

🎉 Finally a release for Laravel 7 support ⛑ Finally I've fixed problems with chunks (Dropzone/Resumable.js with parallel upload). 👍 Also I've found a solution how to get file data after upload for a Dropzone. Check WIKI.

If you created a packagist entry please mark it as deprecated (in favor of this package) or delete it - it can confuse people.

Thank you for your patience.