Disable '?' Concatenation at End of URL When Not Using vod_upstream_extra_args in Remote Mode

[abenmohamed]

Hi,

I am using nginx-auth in remote mode to handle authentication and authorization for requests to a remote service. However, I have encountered an issue where nginx is automatically concatenating a '?' character at the end of the request URL when vod_upstream_extra_args is not used. This behavior is causing problems with request signatures and credentials when interacting with the remote service. When vod_upstream_extra_args is not used, nginx automatically appends a '?' character at the end of the request URL. This behavior is unexpected and is causing signature issues, especially when the URL already contains query parameters, as it results in double '?' characters. I would like to disable this automatic '?' concatenation at the end of the URL when vod_upstream_extra_args is not specified. Instead, I want to have full control over query parameter handling. If there is a known workaround or code modification that can be applied locally to disable this behavior, please provide instructions or code snippets.

Thank you for your help!

[abenmohamed]

@erankor do you have any idea ?

[erankor]

can you share the nginx conf?

[abenmohamed]

@erankor yes , this is my config =>

http { log_format main '$remote_addr $remote_user [$time_local] "$request" ' '$status "$http_referer" "$http_user_agent"';

access_log /dev/stdout main;
error_log stderr debug;

default_type application/octet-stream;
include /usr/local/nginx/conf/mime.types;

# tcp_nopush  on;
tcp_nodelay on;

open_file_cache max=1000 inactive=5m;
open_file_cache_valid 2m;
open_file_cache_min_uses 1;
open_file_cache_errors on;

#aio on;
gzip on;
gzip_types application/vnd.apple.mpegurl;

aws_auth $aws_token {
    access_key ****;
    secret_key ****;
    service s3;
    region ****;
}

aws_auth_presign $aws_presigned $aws_token https://host/$bucketName/$key_name?X-Amz-Expires=600;

server {
    listen 80;
    server_name localhost;

    vod_mode remote;
    vod_upstream_location /remote;
    #vod_upstream_extra_args "pathOnly=1";
    vod_last_modified 'Sun, 19 Nov 2000 08:52:00 GMT';
    vod_last_modified_types *;

    # Change default multibitarate suffix from .urlset to .csmil
    vod_multi_uri_suffix .csmil;
    # vod_cache_buffer_size 8m;

    # vod_metadata_cache                 metadata_cache 2048m;
    # vod_response_cache response_cache  128m;
    vod_last_modified_types *;

    # Fix segment duration to 4s ("vod_align_segments_to_key_frames" should be set to "off" or remove vod_manifest_segment_durations_mode accurate;
    # but may cause problems while playing segments in chrome)
    vod_bootstrap_segment_durations 2000;
    vod_bootstrap_segment_durations 2000;
    vod_segment_duration 4000;
    vod_align_segments_to_key_frames off;
    vod_dash_fragment_file_name_prefix "segment";
    vod_hls_segment_file_name_prefix "segment";

    vod_dash_absolute_manifest_urls off;
    vod_hls_absolute_master_urls off;
    vod_hls_absolute_index_urls off;

    vod_hls_force_unmuxed_segments on;
    # vod_manifest_segment_durations_mode accurate;
    vod_manifest_segment_durations_mode estimate;

    gzip on;
    gzip_types application/vnd.apple.mpegurl video/f4m application/dash+xml text/xml;
    vod_hls_mpegts_align_frames off;
    vod_hls_mpegts_interleave_frames on;

    # Secure s3
    set $temp_token token;

    location ~* /remote/ss/hls/([^/]+) {
        resolver_timeout          20s;
        proxy_pass https://host;
        proxy_redirect off;
        proxy_ssl_server_name on;
        proxy_set_header Host host;
        internal;
        rewrite ^/remote/ss/hls/(?P<bucketName>[^/]+)/(?P<key_name>.*) $aws_presigned break;
        proxy_set_header x-amz-cf-id "";
        proxy_set_header Authorization $aws_token;
        proxy_set_header X-Amz-Date $aws_auth_date;
        proxy_set_header X-Amz-Content-SHA256 *****;
    }

    location ~* ^/ss/hls/([^/]+) {
        vod hls;
        add_header Access-Control-Allow-Headers '*';
        add_header Access-Control-Allow-Origin '*';
        add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
    }

}

}

[rmadrona]

Hello @abenmohamed ,

I don't have a solution to your specific question but I can contribute to solve the problem I have a similar need to authenticate final user on a S3 on-premise storage using Open ID Connect (OIDC) My prototype is not fully functionnal and I had a different approach I wanted to share with you My goal is to decouple streaming for authent to reduce nginx.conf complexity. I am using oauth2-proxy [https://github.com/oauth2-proxy/oauth2-proxy] to delegate tokens managements. I think I am closed to find the solution using a cascade of nginx proxys and I hope I can share with you soon the result of my work.

Regards, R;

[abenmohamed]

Hello @rmadrona ,

Thank you for your response and willingness to contribute to solving the problem. While we appreciate your suggestion to use oauth2-proxy and a cascade of nginx proxies, we have been using a proxy path in our current setup. However, we're exploring an embedded solution that doesn't rely on the proxy path, as it might offer a more efficient approach.

Best regards,

[abenmohamed]

Hello @erankor, I hope you're doing well. I wanted to check in on the status of the issue we discussed earlier. At the moment, we are seeking a temporary solution or workaround rather than a full pull request. Your insights on this matter would be greatly appreciated.

[erankor]

Sorry, took me a while to get to this... I checked it out now -

1. This config doesn't make much sense, you need to decide whether you want to use header authentication or presigned URL, it seems that you're currently doing both. Either remove the proxy_set_headers or the presigned URL. We are using header authentication with nginx-vod-module, so I'd recommend that.
2. If for some reason, you want to use presigned URL, I think the rewrite is causing some trouble here, it seems to escape the ?. Instead of rewrite, you can just use proxy_pass $aws_presigned;, but you will need to specify a resolver in this case.