kaltura / nginx-vod-module

NGINX-based MP4 Repackager
GNU Affero General Public License v3.0
1.99k stars 439 forks source link

mp4_metadata_reader_read: buffer size is smaller than moov size When opening larger files. #1495

Open bruhzinga opened 10 months ago

bruhzinga commented 10 months ago

Hello i am new to Nginx and honestly it scares me.I'm just trying to make a project for uni so i used this docker image from nytimes. I'm using win11 on my main machine. I mapped the videos directory to the docker volume(like in the example they gave ). When i upload small videos everything works fine. But with a large video(2h+) it gives me this error:

2023/12/04 22:03:19 [info] 7#7: 1 client closed connection while waiting for request, client: 172.17.0.1, server: 0.0.0.0:80 2023-12-05 01:03:19 2023/12/04 22:03:19 [error] 7#7: 2 mp4_metadata_reader_read: buffer size 266176 is smaller than moov size 4119062 while reading media header, client: 172.17.0.1, server: localhost, request: "GET /dash/Puss.In.Boots.The.Last.Wish.2022_1080p.mp4,.urlset/manifest.mpd HTTP/1.1", host: "localhost:3030".

The .mp4 file alone plays fine. I am aware of issues already opened but none of them helped me #1316 #1424 #1069 Here is my nginx.conf

worker_processes auto;

events { use epoll; }

http { log_format main '$remote_addr $remote_user [$time_local] "$request" ' '$status "$http_referer" "$http_user_agent"';

access_log /dev/stdout main; error_log stderr debug;

default_type application/octet-stream; include /usr/local/nginx/conf/mime.types;

sendfile on; tcp_nopush on; tcp_nodelay on;

vod_mode local; vod_metadata_cache metadata_cache 16m; vod_response_cache response_cache 2048m; vod_last_modified_types *; vod_segment_duration 9000; vod_align_segments_to_key_frames on; vod_dash_fragment_file_name_prefix "segment"; vod_hls_segment_file_name_prefix "segment";

vod_manifest_segment_durations_mode accurate;

open_file_cache max=1000 inactive=5m; open_file_cache_valid 2m; open_file_cache_min_uses 1; open_file_cache_errors on;

aio on;

server { listen 80; server_name localhost; root /opt/static;

  location ~ ^/videos/.+$ {
      autoindex on;
  }

  location /hls/ {
      vod hls;
      alias /opt/static/videos/;
      add_header Access-Control-Allow-Headers '*';
      add_header Access-Control-Allow-Origin '*';
      add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
  }

  location /thumb/ {
      vod thumb;
      alias /opt/static/videos/;
      add_header Access-Control-Allow-Headers '*';
      add_header Access-Control-Allow-Origin '*';
      add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
  }

  location /dash/ {
      vod dash;
      alias /opt/static/videos/;
      add_header Access-Control-Allow-Headers '*';
      add_header Access-Control-Allow-Origin '*';
      add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
  }

} }

Any help would be greatly appreciated.

maximk-1 commented 10 months ago

try to tune vod_cache_buffer_size and i advice you to increase vod_max_frame_count

bruhzinga commented 10 months ago

try to tune vod_cache_buffer_size and i advice you to increase vod_max_frame_count

Thank you for reaching out . I set vod_cache_buffer_size to 2048m. Didnt fix my problem . Can't set vod_max_frame_count because i get this error

nginx: [emerg] unknown directive "vod_max_frame_count" in /usr/local/nginx/conf/nginx.conf:68

maximk-1 commented 10 months ago

I set vod_cache_buffer_size to 2048m. Didnt fix my problem

also try to tune vod_max_metadata_size

Can't set vod_max_frame_count because i get this error nginx: [emerg] unknown directive "vod_max_frame_count" in /usr/local/nginx/conf/nginx.conf:68

show full config

bruhzinga commented 10 months ago

I set vod_cache_buffer_size to 2048m. Didnt fix my problem

also try to tune vod_max_metadata_size

Can't set vod_max_frame_count because i get this error nginx: [emerg] unknown directive "vod_max_frame_count" in /usr/local/nginx/conf/nginx.conf:68

show full config

Here's my .conf as i said before it can't start because unknown directive "vod_max_frame_count" in /usr/local/nginx/conf/nginx.conf:31

I think i might be using an older version of the module and/or nginx because that docker image hasnt been updated in a year. But still it did work with smaller sized videos. worker_processes auto;

events { use epoll; }

http { log_format main '$remote_addr $remote_user [$time_local] "$request" ' '$status "$http_referer" "$http_user_agent"';

access_log /dev/stdout main; error_log stderr debug;

default_type application/octet-stream; include /usr/local/nginx/conf/mime.types;

sendfile on; tcp_nopush on; tcp_nodelay on;

vod_mode local; vod_metadata_cache metadata_cache 512m; vod_response_cache response_cache 2048m; vod_last_modified_types *; vod_segment_duration 9000; vod_align_segments_to_key_frames on; vod_dash_fragment_file_name_prefix "segment"; vod_hls_segment_file_name_prefix "segment"; vod_cache_buffer_size 2048m; vod_max_metadata_size 2048m; vod_max_frame_count 100;

vod_manifest_segment_durations_mode accurate;

open_file_cache max=1000 inactive=5m; open_file_cache_valid 2m; open_file_cache_min_uses 1; open_file_cache_errors on;

aio on;

server {

  listen 80;
  server_name localhost;
  root /opt/static;

  location ~ ^/videos/.+$ {
      autoindex on;
  }

  location /hls/ {
      vod hls;
      alias /opt/static/videos/;
      add_header Access-Control-Allow-Headers '*';
      add_header Access-Control-Allow-Origin '*';
      add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
  }

  location /thumb/ {
      vod thumb;
      alias /opt/static/videos/;
      add_header Access-Control-Allow-Headers '*';
      add_header Access-Control-Allow-Origin '*';
      add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
  }

  location /dash/ {
      vod dash;
      alias /opt/static/videos/;
      add_header Access-Control-Allow-Headers '*';
      add_header Access-Control-Allow-Origin '*';
      add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
  }

} }

erankor commented 9 months ago

There have been issues reported in the past related to Windows/WSL, for example this one #1424. Since I think a production environment should be using Linux :) I didn't consider it a priority to dig further into this.

bruhzinga commented 9 months ago

Well too bad for me I guess . I already switched to shaka-streamer package in python . The only functionality I'm missing is thumbnail generation and now I needed to write code to serve dash / hls files . And in the end it was only for a university project so not word shattering problem maybe .

erankor commented 9 months ago

Well, I'm sorry, it's all a matter of focus... the goal of this project is to support on-the-fly packaging in production scale, if pre-packaging is suitable for your use case or you don't care about scale, there are certainly other projects that are a better fit.