directus / directus

The Modern Data Stack 🐰 — Directus is an instant REST+GraphQL API and intuitive no-code data collaboration app for any SQL database.
https://directus.io
Other
26.66k stars 3.7k forks source link

Cannot upload files to cloudinary media storage, error: Inconsistent chunk sizes #21248

Open AndriyStankevych opened 5 months ago

AndriyStankevych commented 5 months ago

Describe the Bug

Error dialog appears after 100% loader when trying to upload files bigger that ~11 MB image Following error writes to log

directus-1  | [04:25:46.564] WARN: Couldn't save file 46649b24-386d-4649-88fb-5a5f5d6113cd.mp4
directus-1  | [04:25:46.564] WARN: Can't upload file "46649b24-386d-4649-88fb-5a5f5d6113cd.mp4": Inconsistent chunk sizes
directus-1  |     err: {
directus-1  |       "type": "Error",
directus-1  |       "message": "Can't upload file \"46649b24-386d-4649-88fb-5a5f5d6113cd.mp4\": Inconsistent chunk sizes: Inconsistent chunk sizes",
directus-1  |       "stack":
directus-1  |           Error: Can't upload file "46649b24-386d-4649-88fb-5a5f5d6113cd.mp4": Inconsistent chunk sizes
directus-1  |               at DriverCloudinary.write (file:///directus/node_modules/.pnpm/file+packages+storage-driver-cloudinary_vue@3.3.13/node_modules/@directus/storage-driver-cloudinary/dist/index.js:307:13)
directus-1  |               at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
directus-1  |               at async FilesService.uploadOne (file:///directus/node_modules/.pnpm/file+api_@unhead+vue@1.8.9_pinia@2.1.7_typescript@5.3.3_vue@3.3.13/node_modules/@directus/api/dist/services/files.js:99:17)
directus-1  |               at async Multipart.<anonymous> (file:///directus/node_modules/.pnpm/file+api_@unhead+vue@1.8.9_pinia@2.1.7_typescript@5.3.3_vue@3.3.13/node_modules/@directus/api/dist/controllers/files.js:89:32)
directus-1  |           caused by: Error: Inconsistent chunk sizes
directus-1  |               at DriverCloudinary.uploadChunk (file:///directus/node_modules/.pnpm/file+packages+storage-driver-cloudinary_vue@3.3.13/node_modules/@directus/storage-driver-cloudinary/dist/index.js:333:13)
directus-1  |               at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
directus-1  |               at async file:///directus/node_modules/.pnpm/p-queue@8.0.1/node_modules/p-queue/dist/index.js:187:36
directus-1  |     }

Everything works as expected for files smaller that ~11 MB

To Reproduce

  1. Configure media storage

    STORAGE_LOCATIONS="cloudinary"
    STORAGE_CLOUDINARY_DRIVER="cloudinary"
    STORAGE_CLOUDINARY_CLOUD_NAME="..."
    STORAGE_CLOUDINARY_API_KEY="..."
    STORAGE_CLOUDINARY_API_SECRET="..."
    STORAGE_CLOUDINARY_ACCESS_MODE="public"
  2. Upload file with size more that 11 MB to media storage from file system

  3. Error dialog from description appears

Directus Version

v10.8.3

Hosting Strategy

Self-Hosted (Docker Image)

GiTiFab88 commented 5 months ago

Get exactly same behavior with same "chunk size" error :/

rijkvanzanten commented 5 months ago

Could either of you share a file that's causing this behavior? Wanna make sure we can reproduce this as accurately as possible

AndriyStankevych commented 5 months ago

Here is the video

Issue is reproducible on any video that creates more than one chunk in addition to the latest. Every next chunk starting from second throw this exception.

DonCastillo commented 1 month ago

I'm having the same problem when uploading a PDF file. Any updates on this?