Large video file can't be moved to Fedora (test case #23)

[dflitner]

I found an old closed issue which I'm not sure is related.

I tried uploading a 2.5GB mp4 file during the test sprint on both my local machine and one that Eli had set up on his computer. We of course had to adjust the Drupal settings to allow large files to be uploaded in the first place.

Multiple attempts gave the same results.

It sat spinning for quite a while and then halted with an error message: File upload error. Could not move uploaded file.

Watchdog has two entries:

Upload error. Could not move uploaded file 1984-clio-awards-19117.mp4 to destination fedora://2020-02/1984-clio-awards-19117.mp4.

Warning: move_uploaded_file(): Unable to move '/tmp/phpNGE6s6' to 'fedora://2020-02/1984-clio-awards-19117.mp4' in Drupal\Core\File\FileSystem->moveUploadedFile() (line 85 of /var/www/html/drupal/web/core/lib/Drupal/Core/File/FileSystem.php)

0 /var/www/html/drupal/web/core/includes/bootstrap.inc(600): _drupal_error_handler_real(2, 'move_uploaded_f...', '/var/www/html/d...', 85, Array)

1 [internal function]: _drupal_error_handler(2, 'move_uploaded_f...', '/var/www/html/d...', 85, Array)

2 /var/www/html/drupal/web/core/lib/Drupal/Core/File/FileSystem.php(85): move_uploaded_file('/tmp/phpNGE6s6', 'fedora://2020-0...')

3 /var/www/html/drupal/web/core/modules/file/file.module(1106): Drupal\Core\File\FileSystem->moveUploadedFile('/tmp/phpNGE6s6', 'fedora://2020-0...')

4 /var/www/html/drupal/web/core/modules/file/file.module(926): _file_save_upload_single(Object(Symfony\Component\HttpFoundation\File\UploadedFile), 'field_media_vid...', Array, 'fedora://2020-0...', 0)

5 /var/www/html/drupal/web/core/modules/file/file.module(813): file_save_upload('field_media_vid...', Array, 'fedora://2020-0...', NULL, 0)

6 /var/www/html/drupal/web/core/modules/file/file.module(1444): _file_save_upload_from_form(Array, Object(Drupal\Core\Form\FormState))

7 /var/www/html/drupal/web/core/modules/file/src/Element/ManagedFile.php(74): file_managed_file_save_upload(Array, Object(Drupal\Core\Form\FormState))

8 /var/www/html/drupal/web/core/modules/file/src/Plugin/Field/FieldWidget/FileWidget.php(324): Drupal\file\Element\ManagedFile::valueCallback(Array, Array, Object(Drupal\Core\Form\FormState))

9 [internal function]: Drupal\file\Plugin\Field\FieldWidget\FileWidget::value(Array, Array, Object(Drupal\Core\Form\FormState))

10 /var/www/html/drupal/web/core/lib/Drupal/Core/Form/FormBuilder.php(1256): call_user_func_array(Array, Array)

11 /var/www/html/drupal/web/core/lib/Drupal/Core/Form/FormBuilder.php(992): Drupal\Core\Form\FormBuilder->handleInputElement('media_video_add...', Array, Object(Drupal\Core\Form\FormState))

12 /var/www/html/drupal/web/core/lib/Drupal/Core/Form/FormBuilder.php(1062): Drupal\Core\Form\FormBuilder->doBuildForm('media_video_add...', Array, Object(Drupal\Core\Form\FormState))

13 /var/www/html/drupal/web/core/lib/Drupal/Core/Form/FormBuilder.php(1062): Drupal\Core\Form\FormBuilder->doBuildForm('media_video_add...', Array, Object(Drupal\Core\Form\FormState))

14 /var/www/html/drupal/web/core/lib/Drupal/Core/Form/FormBuilder.php(1062): Drupal\Core\Form\FormBuilder->doBuildForm('media_video_add...', Array, Object(Drupal\Core\Form\FormState))

15 /var/www/html/drupal/web/core/lib/Drupal/Core/Form/FormBuilder.php(563): Drupal\Core\Form\FormBuilder->doBuildForm('media_video_add...', Array, Object(Drupal\Core\Form\FormState))

16 /var/www/html/drupal/web/core/lib/Drupal/Core/Form/FormBuilder.php(320): Drupal\Core\Form\FormBuilder->processForm('media_video_add...', Array, Object(Drupal\Core\Form\FormState))

17 /var/www/html/drupal/web/core/lib/Drupal/Core/Controller/FormController.php(91): Drupal\Core\Form\FormBuilder->buildForm(Object(Drupal\media\MediaForm), Object(Drupal\Core\Form\FormState))

18 [internal function]: Drupal\Core\Controller\FormController->getContentResult(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Routing\RouteMatch))

19 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array)

20 /var/www/html/drupal/web/core/lib/Drupal/Core/Render/Renderer.php(573): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber{closure}()

21 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure))

22 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array)

23 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(151): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber{closure}()

24 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(68): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1)

25 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/Session.php(57): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)

26 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(47): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)

27 /var/www/html/drupal/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)

28 /var/www/html/drupal/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true)

29 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(47): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)

30 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(52): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)

31 /var/www/html/drupal/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)

32 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(694): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)

33 /var/www/html/drupal/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))

34 {main}

[dannylamb]

I'm surprised you're even allowed to upload it. Isn't 2GB the PHP upload limit?

[elizoller]

Locally, I upped the php_memory_limit to 8GB. In your test VM it looked it was still set to 32MB I believe (which is why we turned to my local install).

[dannylamb]

@elizoller I was thinking of upload_max_filesize. I did some quick googling to confirm my previous statements, and maybe I'm showing my age. The sneaky 2GB max size may have been just a 32-bit thing :older_man:

Regardless, I'm adding this to the milestone. I'll play around with some large files on my end and report back.

[elizoller]

locally i have upload_max_filesize set to 8GB as well

[dannylamb]

After bumping my upload and post max sizes and php memory, I run into this as well with a 2.4 GB binary.

[dannylamb]

@ajstanley You did some work on uploading large binaries for UPEI, do you recall what the limits were before needing to use something like tus or plupload?

[mjordan]

Could PHP's max execution time setting have anything to do with this? In other words, the file is so large, PHP is timing out.

[dflitner]

I wondered that too but wasn't sure if that would throw a time out error rather than couldn't transfer.

---

From: Mark Jordan notifications@github.com Sent: Wednesday, February 12, 2020 10:04:43 AM To: Islandora/documentation documentation@noreply.github.com Cc: Debbie Flitner Debbie.Flitner@asu.edu; Author author@noreply.github.com Subject: Re: [Islandora/documentation] Large video file can't be moved to Fedora (test case #23) (#1436)

Could PHP's max execution time setting have anything to do with this? In other words, the file is so large, PHP is timing out.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Islandora_documentation_issues_1436-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DABT55PHLZLOOYVIBFC32Z3LRCQT2XA5CNFSM4KSVZ34KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELRR3TY-23issuecomment-2D585309647&d=DwMCaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=8pu129jWYvXZTk5QUmlc2Qrw7pqFHuc9hOgN4Nf50Yc&m=ISTq7te5omYPJIq4tN4ocSGn0cndctbeR5vUp_8-m-8&s=WDuIvkXCPgRZi1xoeEy_JChMi8rSToxn4vZHJWB_asY&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABT55PDRZB5Y7Y6S5SXFZS3RCQT2XANCNFSM4KSVZ34A&d=DwMCaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=8pu129jWYvXZTk5QUmlc2Qrw7pqFHuc9hOgN4Nf50Yc&m=ISTq7te5omYPJIq4tN4ocSGn0cndctbeR5vUp_8-m-8&s=V8yKIBQIgyyPsH6j5xHqdB5FWcPoaU04ahHKFhuQ1Rs&e=.

[elizoller]

I'll try it.

[elizoller]

I upped the max execution time to 12000 and then tried a 3.88GB file and got "The file could not be saved. An unknown error has occurred." with no further message in the recent logs.

[dflitner]

Could it be a problem with the available memory for tmp files? I'm not sure if Islandora has tweaked the process but, generally, a file will upload completely to the local Drupal tmp folder and then transfer to the endpoint. Is it just that our VBs don't have space for big files?

I fixed the problem on another site by making sure the Drupal directory/partition had a lot of available space and using File Resumable Upload to help.

[elizoller]

^ That idea came up in the tech call yesterday. I think @dannylamb was going to try one on a hosted VM instead of the Vagrant VM to hopefully surpass issues like that

[dannylamb]

I've tried three times to upload a big file to http://138.197.139.139 and it just results in an indefinitely spinning wheel :(

[dannylamb]

Yeah, still doing this. Tried again today and ended with another endlessly spinning wheel. Maybe someone with something better than my country internets can try.

[dflitner]

I tried uploading to your sure and also got a spinning wheel for a long time and then the upload silently failed.

---

From: dannylamb notifications@github.com Sent: Tuesday, February 18, 2020 7:29:52 PM To: Islandora/documentation documentation@noreply.github.com Cc: Debbie Flitner Debbie.Flitner@asu.edu; Author author@noreply.github.com Subject: Re: [Islandora/documentation] Large video file can't be moved to Fedora (test case #23) (#1436)

Yeah, still doing this. Tried again today and ended with another endlessly spinning wheel. Maybe someone with something better than my country internets can try.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Islandora_documentation_issues_1436-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DABT55PGUHWHLEAF575EKIJLRDSKSBA5CNFSM4KSVZ34KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMGDRAA-23issuecomment-2D588003456&d=DwMCaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=8pu129jWYvXZTk5QUmlc2Qrw7pqFHuc9hOgN4Nf50Yc&m=I0XvKEt48Q2hL4a5PIPmI2R8oq8DuqAi7_k2SAySXZM&s=47SytzOYrAkRS7qZqR97OpIIV8a8Ala6Qv6nEpOaHYw&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABT55PAOW7Y5LJDP44F3KKLRDSKSBANCNFSM4KSVZ34A&d=DwMCaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=8pu129jWYvXZTk5QUmlc2Qrw7pqFHuc9hOgN4Nf50Yc&m=I0XvKEt48Q2hL4a5PIPmI2R8oq8DuqAi7_k2SAySXZM&s=TP9v0Tnxy1wAuJJS3txtp3GhMyaAJUgvSD1rhS66AxY&e=.

[elizoller]

Do you mean another tester that is pointing your VM or a different VM?

[dannylamb]

I think @dflitner just recreated what I've been experiencing. I was hoping for better behaviour from a remote environment, but it's worse :sob:

At this point in the very least we should confirm the max size before it starts to crap out so we can state it as a known issue. E.g. "If you're uploading a file bigger than 2GB, consider using plupload or migrate_islandora_csv" in the release notes/documentation/UI. FWIW in Islandora 7 you've got to do pretty much the same. Either it's plupload or a custom batch for files bigger than 2GB.

I'm thinking @ajstanley and @alxp may have some insight on large files from their work at UPEI.

[dannylamb]

After a recent conversation, it seems most likely that this is due to running small vagrant evnironments on our laptops. @dflitner's done it on D8 sites and @alxp has done it on an I8 site, but those are both properly configured production environments. Chances are the vagrant environment just doesn't have the resources to do something in the GB range of file size. So maybe the best we can do is just document what you need to keep in mind when configuring a live site. Is that an acceptable outcome to this issue?

In general, though, large files over HTTP are generally problematic, and we should investigate using @ajstanley's TUS module for the UI. Also, @wgillingham has successfully done large files over FTP in I7, so we should probably check out the FTP plugins for the migrate API too.