Video file uploads in Nova don't work consistently

connecteev commented 1 year ago

Laravel Version: 10.22
Nova Version: 4.27.12 (Silver Surfer)
PHP Version: 8.2
Database Driver & Version: MySql 8
Operating System and Version: Mac OS Monterey 12.x
Browser type and version: Chrome Version 116.0.5845.179 (Official Build) (arm64)

Description:

I am having a very frustrating time with Nova File uploads. Most file uploads dont work consistently. Nova doesn't seem to know of / handle errors very gracefully, and file uploads don't seem to work that well. This is despite me perusing / trying everything I could find in the File documentation.

I have tested with https://file-examples.com/index.php/sample-video-files/sample-mov-files-download/ and https://file-examples.com/index.php/sample-video-files/sample-mp4-files/ So far, I have only managed to get specific files to work:

1.4MB mov file: works fine
2.2MB mov file: fails
1.5MB MP4 file: works fine
3MB MP4 file: fails

Detailed steps to reproduce the issue on a fresh Nova installation:

Create a CourseVideo Nova resource with the following File field type to accept file / video uploads

File::make('Video Path', 'video_path')
                ->disk('public')
                ->acceptedTypes('video/*')
                ->store(function (Request $request, $model) {
                    return [
                        'video_path' => $request->file('video_path')->store('attachments'),
                    ];
                })
                ->sortable()
                ->required() // alternative to using 'required' in the rules() method
                //->rules('required', 'string', 'max:255')
            ,

course_videos migration file:

    public function up(): void
    {
        Schema::create('course_videos', function (Blueprint $table) {
            $table->id();
            $table->string('title');
            $table->string('slug')->unique();
            $table->text('description');
            $table->string('video_path');
            $table->timestamps();
        });
    }

Now, go to the Nova admin and try to upload a new video, in my case: /admin/resources/course-videos/new On submitting the form (in my case pressing the "Create Course Video" button), you either see:

Case 1: a success message (as shown in the screenshot below), there is an entry added to the database and the file upload worked properly (this happens for the two 1.5 MB files: MOV and MP4, as listed above)
Case 2: or, you see a success message (as shown in the screenshot below), BUT there is no entry added to the database and the file upload did not work
Case 3: or, you see the "Server error" and "There was a problem submitting the form" error messages (as shown in the screenshot below)

Case 1 (File upload works successfully):

Case 2 (File upload fails, but Nova thinks it worked):

Case 3 (File upload fails, Nova gives an error message):

In both case 2 and 3, this is what you see a similar error in the laravel.log server logs:

[2023-09-11 15:45:59] local.ERROR: SQLSTATE[HY000]: General error: 1364 Field 'video_path' doesn't have a default value (Connection: mysql, SQL: insert into `course_videos` (`title`, `slug`, `description`, `updated_at`, `created_at`) values (This is a test video, this-is-a-test-video, <div>Testing video uploads in Nova</div>, 2023-09-11 15:45:59, 2023-09-11 15:45:59)) {"userId":1,"exception":"[object] (Illuminate\\Database\\QueryException(code: HY000): SQLSTATE[HY000]: General error: 1364 Field 'video_path' doesn't have a default value (Connection: mysql, SQL: insert into `course_videos` (`title`, `slug`, `description`, `updated_at`, `created_at`) values (This is a test video, this-is-a-test-video, <div>Testing video uploads in Nova</div>, 2023-09-11 15:45:59, 2023-09-11 15:45:59)) at /Code/web/__Herd/inertia_app/vendor/laravel/framework/src/Illuminate/Database/Connection.php:801)

So, it looks like 1.4MB MOV and MP4 files work, but 2MB+ files don't. It has to be an issue with my server settings (Nginx, Apache, etc), right? I looked at the responses at https://github.com/laravel/nova-issues/issues/2527 and https://github.com/laravel/nova-issues/issues/1998 and checked my server settings. They seem fine. Here are snippets from my settings for Nginx and Laravel Herd. It looks like nginx.conf includes herd.conf. The values for client_max_body_size are 512M and 128M respectively.

What is going on? This has me stumped, and I wish Nova handled these essentials more gracefully.

nginx.conf:

user "connecteev" staff;
worker_processes auto;

events {
    worker_connections  1024;
}

http {

types {
    text/html                                        html htm shtml;
    text/css                                         css;
    text/xml                                         xml;
    image/gif                                        gif;
    image/jpeg                                       jpeg jpg;
    application/javascript                           js;
    application/atom+xml                             atom;
    application/rss+xml                              rss;

    text/mathml                                      mml;
    text/plain                                       txt;
    text/vnd.sun.j2me.app-descriptor                 jad;
    text/vnd.wap.wml                                 wml;
    text/x-component                                 htc;

    image/avif                                       avif;
    image/png                                        png;
    image/svg+xml                                    svg svgz;
    image/tiff                                       tif tiff;
    image/vnd.wap.wbmp                               wbmp;
    image/webp                                       webp;
    image/x-icon                                     ico;
    image/x-jng                                      jng;
    image/x-ms-bmp                                   bmp;

    font/woff                                        woff;
    font/woff2                                       woff2;

    application/java-archive                         jar war ear;
    application/json                                 json;
    application/mac-binhex40                         hqx;
    application/msword                               doc;
    application/pdf                                  pdf;
    application/postscript                           ps eps ai;
    application/rtf                                  rtf;
    application/vnd.apple.mpegurl                    m3u8;
    application/vnd.google-earth.kml+xml             kml;
    application/vnd.google-earth.kmz                 kmz;
    application/vnd.ms-excel                         xls;
    application/vnd.ms-fontobject                    eot;
    application/vnd.ms-powerpoint                    ppt;
    application/vnd.oasis.opendocument.graphics      odg;
    application/vnd.oasis.opendocument.presentation  odp;
    application/vnd.oasis.opendocument.spreadsheet   ods;
    application/vnd.oasis.opendocument.text          odt;
    application/vnd.openxmlformats-officedocument.presentationml.presentation
                                                     pptx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
                                                     xlsx;
    application/vnd.openxmlformats-officedocument.wordprocessingml.document
                                                     docx;
    application/vnd.wap.wmlc                         wmlc;
    application/wasm                                 wasm;
    application/x-7z-compressed                      7z;
    application/x-cocoa                              cco;
    application/x-java-archive-diff                  jardiff;
    application/x-java-jnlp-file                     jnlp;
    application/x-makeself                           run;
    application/x-perl                               pl pm;
    application/x-pilot                              prc pdb;
    application/x-rar-compressed                     rar;
    application/x-redhat-package-manager             rpm;
    application/x-sea                                sea;
    application/x-shockwave-flash                    swf;
    application/x-stuffit                            sit;
    application/x-tcl                                tcl tk;
    application/x-x509-ca-cert                       der pem crt;
    application/x-xpinstall                          xpi;
    application/xhtml+xml                            xhtml;
    application/xspf+xml                             xspf;
    application/zip                                  zip;

    application/octet-stream                         bin exe dll;
    application/octet-stream                         deb;
    application/octet-stream                         dmg;
    application/octet-stream                         iso img;
    application/octet-stream                         msi msp msm;

    audio/midi                                       mid midi kar;
    audio/mpeg                                       mp3;
    audio/ogg                                        ogg;
    audio/x-m4a                                      m4a;
    audio/x-realaudio                                ra;

    video/3gpp                                       3gpp 3gp;
    video/mp2t                                       ts;
    video/mp4                                        mp4;
    video/mpeg                                       mpeg mpg;
    video/quicktime                                  mov;
    video/webm                                       webm;
    video/x-flv                                      flv;
    video/x-m4v                                      m4v;
    video/x-mng                                      mng;
    video/x-ms-asf                                   asx asf;
    video/x-ms-wmv                                   wmv;
    video/x-msvideo                                  avi;
}

    default_type  application/octet-stream;

    sendfile on;
    keepalive_timeout  65;
    types_hash_max_size 2048;

    client_max_body_size 512M;

    server_names_hash_bucket_size 128;

    gzip  on;
    gzip_comp_level 5;
    gzip_min_length 256;
    gzip_proxied any;
    gzip_vary on;
    gzip_types
    application/atom+xml
    application/javascript
    application/json
    application/rss+xml
    application/vnd.ms-fontobject
    application/x-font-ttf
    application/x-web-app-manifest+json
    application/xhtml+xml
    application/xml
    font/opentype
    image/svg+xml
    image/x-icon
    text/css
    text/plain
    text/x-component;

    include "/Users/connecteev/Library/Application Support/Herd/config/valet/Nginx/*";
    include herd.conf;
}

herd.conf

server {
    listen 127.0.0.1:80 default_server;
    #listen VALET_LOOPBACK:80; # valet loopback
    root /;
    charset utf-8;
    client_max_body_size 128M;

    location /41c270e4-5535-4daa-b23e-c269744c2f45/ {
        internal;
        alias /;
        try_files $uri $uri/;
    }

    location / {
        rewrite ^ "/Applications/Herd.app/Contents/Resources/valet/server.php" last;
    }

    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }

    access_log off;
    error_log "/Users/connecteev/Library/Application Support/Herd/Log/nginx-error.log";

    error_page 404 "/Applications/Herd.app/Contents/Resources/valet/server.php";

    location ~ [^/]\.php(/|$) {
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass "unix:/Users/connecteev/Library/Application Support/Herd/herd.sock";
        fastcgi_index "/Applications/Herd.app/Contents/Resources/valet/server.php";
        fastcgi_param QUERY_STRING  $query_string;
        fastcgi_param REQUEST_METHOD  $request_method;
        fastcgi_param CONTENT_TYPE  $content_type;
        fastcgi_param CONTENT_LENGTH  $content_length;
        fastcgi_param SCRIPT_FILENAME  $request_filename;
        fastcgi_param SCRIPT_NAME  $fastcgi_script_name;
        fastcgi_param REQUEST_URI  $request_uri;
        fastcgi_param DOCUMENT_URI  $document_uri;
        fastcgi_param DOCUMENT_ROOT  $document_root;
        fastcgi_param SERVER_PROTOCOL  $server_protocol;
        fastcgi_param GATEWAY_INTERFACE CGI/1.1;
        fastcgi_param SERVER_SOFTWARE  nginx/$nginx_version;
        fastcgi_param REMOTE_ADDR  $remote_addr;
        fastcgi_param REMOTE_PORT  $remote_port;
        fastcgi_param SERVER_ADDR  $server_addr;
        fastcgi_param SERVER_PORT  $server_port;
        fastcgi_param SERVER_NAME  $server_name;
        fastcgi_param HTTPS   $https if_not_empty;
        fastcgi_param HERD_HOME "/Users/connecteev/Library/Application Support/Herd";
        fastcgi_param REDIRECT_STATUS  200;
        fastcgi_param HTTP_PROXY  "";
        fastcgi_buffer_size 512k;
        fastcgi_buffers 16 512k;
        fastcgi_param SCRIPT_FILENAME "/Applications/Herd.app/Contents/Resources/valet/server.php";
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }

    location ~ /\.ht {
        deny all;
    }
}

crynobone commented 1 year ago

What's the configured max_post_size set on PHP?

connecteev commented 1 year ago

@crynobone thanks..I am now able to upload a max file size of 350MB, using these changes:

/Users/connecteev/Library/Application Support/Herd/config/php/{74,80,81,82,83}/php.ini: Added to all php.ini files:

upload_max_filesize=1G
post_max_size=1G

/Users/connecteev/Library/Application Support/Herd/config/nginx/nginx.conf: Changed client_max_body_size 512M; to client_max_body_size 1G;

/Users/connecteev/Library/Application Support/Herd/config/nginx/herd.conf: Changed client_max_body_size 128M; to client_max_body_size 1G;

Then restarted Herd (php and nginx).

I can now upload a 350MB file, though 600MB-950MB uploads still fail. There is nothing in the laravel error log, and this is the error coming from Nginx now:

2023/09/11 17:48:44 [error] 14243#3547640: *179 client intended to send too large body: 939929681 bytes, client: 127.0.0.1, server: inertia.test, request: "POST /nova-api/course-videos?editing=true&editMode=create HTTP/2.0", host: "inertia.test", referrer: "https://inertia.test/admin/resources/course-videos/new"

(Nova relays There was a problem submitting the form. to the front-end in this block of code: /vandor/laravel/nova/resources/js/mixins/HandlesFormRequest.js:

    handleResponseError(error) {
      if (error.response === undefined || error.response.status == 500) {
        Nova.error(this.__('There was a problem submitting the form.'))
      } else if (error.response.status == 422) {
        this.validationErrors = new Errors(error.response.data.errors)
        Nova.error(this.__('There was a problem submitting the form.'))
      } else {
        Nova.error(
          this.__('There was a problem submitting the form.') +
            ' "' +
            error.response.statusText +
            '"'
        )
      }
    },

So, while I still want to raise the limit to be able to upload ~1GB files and haven't been able to figure out how to do that, closing this as it is not a Nova issue but an Nginx / PHP issue.

connecteev commented 1 year ago

From more digging: https://www.nginx.com/resources/wiki/modules/upload/#upload-max-file-size For “hard” limit client_max_body_size directive must be used. The value of zero for this directive specifies that no restrictions on file size should be applied.

Interestingly, I tried setting client_max_body_size 0 in both the nginx.conf and herd.conf, but that didnt change anything. Still cant seem to upload beyond 350MB files, and I wonder why nginx doesn't like that. Very strange.

connecteev commented 1 year ago

@crynobone just realized one thing - it would be nice if Nova handled the errors like this more elegantly. Lmk if you'd like to reopen this, want a separate ticket, or have some other preference.

Of course, if you have thoughts on how to upload larger files, that would be helpful also.

laravel / nova-issues