kaltura / nginx-vod-module

NGINX-based MP4 Repackager
GNU Affero General Public License v3.0
1.99k stars 439 forks source link

Remote fallback + vod_metadata_cache problem #1447

Closed jhemmmm closed 1 year ago

jhemmmm commented 1 year ago

Hello,

I have been having an issue with remote fallback.

Enabling the vod_metadata_cache, even if the request from the remote URL has a problem it still returns 200 return code.

Causing the remove fallback wont work.

   vod_cache_buffer_size 8m;
   vod_metadata_cache metadata_cache 512m;

   location /remote_proxy/hls/ {
      internal;
      proxy_pass http://cdn1.example.com/video/;
   }

   location /fallback_remote_proxy/fallback_hls/ {
      internal;
      proxy_pass http://cdn2.example.com/video_fallback_backup/;
   }

   location /fallback_hls/ {
      vod hls;
      vod_mode remote;
      vod_upstream_location /fallback_remote_proxy;
   }

   location /hls/ {
      vod hls;
      vod_mode remote;
      vod_upstream_location /remote_proxy;
      error_page 502 = @errpage;

      add_header Access-Control-Allow-Headers '*';
      add_header Access-Control-Expose-Headers 'Server,range,Content-Length,Content-Range';
      add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
      add_header Access-Control-Allow-Origin '*';
   }

   location @errpage {
      rewrite ^/hls/(.*) /fallback_hls/$1 last;
   }
erankor commented 1 year ago

If you request a manifest (m3u8) and the video metadata is already in cache, the server can build the response without reading anything from upstream and you get 200. If what you meant is that segment requests are failing when there is a problem with the upstream, and not using the fallback - this makes sense. This feature was developed to handle cases where the video file is missing, it wasn't built to handle upstream unavailability. Consider, for example, a multi-datacenter setup where a video is uploaded to one datacenter. It takes time until it is synched to the other datacenters, and during this period a request may arrive to a datacenter that doesn't have the file yet. This is where the fallback feature can help, by proxying the request to a server that has the video.

jhemmmm commented 1 year ago

If you request a manifest (m3u8) and the video metadata is already in cache, the server can build the response without reading anything from upstream and you get 200. If what you meant is that segment requests are failing when there is a problem with the upstream, and not using the fallback - this makes sense. This feature was developed to handle cases where the video file is missing, it wasn't built to handle upstream unavailability. Consider, for example, a multi-datacenter setup where a video is uploaded to one datacenter. It takes time until it is synched to the other datacenters, and during this period a request may arrive to a datacenter that doesn't have the file yet. This is where the fallback feature can help, by proxying the request to a server that has the video.

That makes sense. That problem I was having was an upstream problem when the upstream URL got automatically closed for some reason like an internal server error.