ProcessScriptVariables does not work

[nikolay]

The Nginx configuration snippet:

http {
    pagespeed ProcessScriptVariables on;
    server {
        pagespeed LoadFromFileMatch "^https?://$host/" "$document_root/";
    }
}

The error I get is:

invalid variable name in /etc/nginx/nginx.conf:874
"pagespeed" directive "LoadFromFileRuleMatch" Failed to compile script variables in /etc/nginx/nginx.conf:874

[oschaaf]

@nikolay I can't reproduce that using a simple configuration that includes the lines you mention. The error message implicates that ngx_http_script_compile() indicated failure [1].

Are you sure that pagespeed LoadFromFileMatch "^https?://$host/" "$document_root/"; is at line number 874 in nginx.conf, and is the one that fails?

If so, I'd love to have your (anonymized) configuration to help reproduce.

[1] https://github.com/pagespeed/ngx_pagespeed/blob/master/src/ngx_rewrite_options.cc#L316

[nikolay]

@oschaaf Yes, exactly. Sorry for the confusion, but my configuration is huge and I just wanted to include the relevant settings.

[oschaaf]

I tried a few things, but I still can't reproduce the error message. To speed up reproduction/debugging, would it be possible to forward me an (anonymized) copy of the failing nginx.conf?

[nikolay]

@oschaaf I will email it to you.

[dannymichel]

Same error

user www-data;
worker_processes 4;
pid /run/nginx.pid;

events {
    worker_connections 768;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    fastcgi_cache_path /usr/local/nginx-cache levels=1:2 keys_zone=phpcache:100m inactive=60m;
    fastcgi_cache_key "$scheme$request_method$host$request_uri";  
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request "'
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;
    pagespeed on;
    pagespeed CreateSharedMemoryMetadataCache "/var/ngx_pagespeed_cache" 51200;
    pagespeed ProcessScriptVariables on;
    pagespeed FetchWithGzip on;
    pagespeed DisableRewriteOnNoTransform off;
    pagespeed SupportNoScriptEnabled false;
    pagespeed ListOutstandingUrlsOnError on;
    pagespeed FileCachePath /var/ngx_pagespeed_cache;
    pagespeed FileCacheSizeKb            102400;
    pagespeed FileCacheCleanIntervalMs   3600000;
    pagespeed FileCacheInodeLimit        500000;
    pagespeed LRUCacheKbPerProcess     1024;
    pagespeed LRUCacheByteLimit        16384;
    pagespeed MemcachedThreads 1;
    pagespeed MemcachedServers "localhost:11211";
    pagespeed EnableCachePurge on;
    pagespeed PurgeMethod PURGE;
    pagespeed DownstreamCacheRewrittenPercentageThreshold 95;
    pagespeed MaxCacheableContentLength -1;
    pagespeed EnableFilters add_instrumentation;
    pagespeed EnableFilters insert_dns_prefetch;
    pagespeed EnableFilters extend_cache;
    pagespeed EnableFilters extend_cache_pdfs;
    pagespeed EnableFilters local_storage_cache;
    pagespeed ImplicitCacheTtlMs 31536000000;
    pagespeed LoadFromFileCacheTtlMs 31536000000;
    pagespeed FetchHttps enable,allow_self_signed,allow_unknown_certificate_authority,allow_certificate_not_yet_valid;
    pagespeed RewriteLevel CoreFilters;
    pagespeed JsInlineMaxBytes 10485760;
    pagespeed EnableFilters combine_heads;
    pagespeed EnableFilters collapse_whitespace;
    pagespeed EnableFilters convert_meta_tags;
    pagespeed EnableFilters elide_attributes;
    pagespeed EnableFilters pedantic;
    pagespeed EnableFilters remove_comments;
    pagespeed EnableFilters remove_quotes;
    pagespeed EnableFilters trim_urls;
    pagespeed EnableFilters combine_javascript;
    pagespeed EnableFilters canonicalize_javascript_libraries;
    pagespeed EnableFilters inline_javascript;
    pagespeed EnableFilters outline_css;
    pagespeed EnableFilters combine_css;
    pagespeed EnableFilters inline_import_to_link;
    pagespeed EnableFilters inline_css;
    pagespeed EnableFilters inline_google_font_css;
    pagespeed EnableFilters move_css_above_scripts;
    pagespeed EnableFilters move_css_to_head;
    pagespeed EnableFilters prioritize_critical_css;
    pagespeed EnableFilters rewrite_css;
    pagespeed EnableFilters fallback_rewrite_css_urls;
    pagespeed EnableFilters rewrite_style_attributes_with_url;
    pagespeed EnableFilters dedup_inlined_images;
    pagespeed EnableFilters inline_preview_images;
    pagespeed EnableFilters resize_mobile_images;
    pagespeed EnableFilters lazyload_images;
    pagespeed EnableFilters inline_images;
    pagespeed EnableFilters convert_gif_to_png;
    pagespeed EnableFilters convert_jpeg_to_progressive;
    pagespeed EnableFilters recompress_jpeg;
    pagespeed EnableFilters recompress_png;
    pagespeed EnableFilters recompress_webp;
    pagespeed EnableFilters strip_image_color_profile;
    pagespeed EnableFilters strip_image_meta_data;
    pagespeed EnableFilters jpeg_subsampling;
    pagespeed EnableFilters convert_png_to_jpeg;
    pagespeed EnableFilters resize_images;
    pagespeed EnableFilters resize_rendered_image_dimensions;
    pagespeed EnableFilters convert_jpeg_to_webp;
    pagespeed EnableFilters convert_to_webp_lossless;
    pagespeed EnableFilters insert_image_dimensions;
    pagespeed NoTransformOptimizedImages on;
    pagespeed EnableFilters insert_ga;
    add_header Vary "Accept-Encoding";
    add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
    add_header X-Frame-Options SAMEORIGIN;
    add_header X-Content-Type-Options nosniff;
    add_header X-XSS-Protection "1; mode=block";
    sendfile  off;
    autoindex off;
    map $scheme $fastcgi_https { ## Detect when HTTPS is used
        default off;
        https on;
    }

    keepalive_timeout  10;

    gzip  on;
    gzip_comp_level 2;
    gzip_proxied any;
    gzip_types text/css text/x-component application/x-javascript application/javascript text/javascript text/x-js text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/bmp application/java application/msword application/vnd.ms-fontobject application/x-msdownload image/x-icon image/webp application/json application/vnd.ms-access application/vnd.ms-project application/x-font-otf application/vnd.ms-opentype application/vnd.oasis.opendocument.database application/vnd.oasis.opendocument.chart application/vnd.oasis.opendocument.formula application/vnd.oasis.opendocument.graphics application/vnd.oasis.opendocument.spreadsheet application/vnd.oasis.opendocument.text audio/ogg application/pdf application/vnd.ms-powerpoint application/x-shockwave-flash image/tiff application/x-font-ttf audio/wav application/vnd.ms-write application/font-woff application/font-woff2 application/vnd.ms-excel;
    gzip_min_length  1100;
    gzip_buffers  4 32k;
    gzip_vary on;

    include /usr/local/nginx/sites-enabled/*;
    client_max_body_size 60M;  
}

server {
    listen 80;
    listen [::]:80;
    server_name domain.com www.domain.com;
    rewrite ^ https://domain.com$request_uri? permanent;
}
server {
    listen 443 ssl;
    listen [::]:443 ssl default_server ipv6only=on;
    pagespeed FetchHttps enable,allow_self_signed;
    pagespeed SslCertDirectory /usr/local/nginx/ssl;
    pagespeed MapOriginDomain "http://localhost" "https://domain.com";
    pagespeed LoadFromFileMatch "^https?://domain.com/" "/home/domain/public_html/";
    pagespeed LoadFromFileRuleMatch disallow .*;
    pagespeed LoadFromFileRuleMatch allow \.(ico|pdf|swf|eot|woff|ttf|otf|css|js|jpeg|jpg|png|gif|svg|svgz|mpg|mpeg|mp3|m4a|m4v|mp4|ogg|wmv|mov|mng|3gpp|3gp|webp|webm|flv|avi|asx|asf)(\?.*)?$;
    pagespeed AnalyticsID UA-xxxxxxx-1;
    ssl_certificate     /usr/local/nginx/ssl/domain.com.crt;
    ssl_certificate_key /usr/local/nginx/ssl/domain.key;
    ssl_dhparam /usr/local/nginx/ssl/dhparam.pem;
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /usr/local/nginx/ssl/domain.com.ca-bundle;
    resolver 8.8.8.8 8.8.4.4;
    ssl_session_cache shared:SSL:20m;
    ssl_session_timeout 180m;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK';
    server_name domain.com;
    root /home/domain/public_html;
    set $no_cache 0;
    location /admin {
      set $no_cache 1;
    }
    access_log off;
    error_log off;
    location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" {
      add_header "" "";
    }
    location ~ "^/pagespeed_static/" { }
    location ~ "^/ngx_pagespeed_beacon$" { }
    location = /favicon.ico {
        try_files $uri =204;
    }
    location / {
        index index.html index.php;
        try_files $uri $uri/ @handler;
        expires 7d;
    }
    location ~* \.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|eot|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|webp|json|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|otf|_otf|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|svg|svgz|swf|tar|tif|tiff|ttf|ttc|_ttf|wav|wma|wri|woff|woff2|xla|xls|xlsx|xlt|xlw|zip)$ {
      expires 365d;
    }
    location /shop {
        try_files $uri /shop/index.php;
    }
    location /shop/downloader {
        try_files $uri /shop/downloader/index.php;
    }
    location ~ ^/(app|includes|media/downloadable|pkginfo|report/config.xml|var)/ { deny all; }
    location ~ ^/(info.php|var/export/)/ {
        auth_basic "Restricted Access";
        auth_basic_user_file /etc/nginx/htpasswd;
    }
    location @handler {
        rewrite / /index.php;
    }
    location ~ .php/ {
        rewrite ^(.*.php)/ $1 last;
    }
    location ~ .php$ {
        add_header Vary "Accept-Encoding";
        add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
        add_header X-Frame-Options SAMEORIGIN;
        add_header X-Content-Type-Options nosniff;
        add_header X-XSS-Protection "1; mode=block";
        fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
        fastcgi_index index.php;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param SCRIPT_NAME $fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
        fastcgi_param MAGE_RUN_TYPE website;
        fastcgi_param MAGE_RUN_CODE base;
        fastcgi_cache phpcache; # The name of the cache key-zone to use
          fastcgi_cache_valid 200 30m; # What to cache: 'code 200' responses, for half an hour
          fastcgi_cache_methods GET HEAD; # What to cache: only GET and HEAD requests (ot POST)
          add_header X-Fastcgi-Cache $upstream_cache_status; # Allow us to see if the cache was HIT, MISS, or BYPASSED inside a browser's Inspector panel
          fastcgi_cache_bypass $no_cache; # Dont pull from the cache if true
          fastcgi_no_cache $no_cache; # Dont save to the cache if true
          include /usr/local/nginx/conf/fastcgi_params;
    }
    location ~ ([^/]*)sitemap(.*)\.x(m|s)l$ {
        rewrite ^/sitemap\.xml$ /sitemap_index.xml permanent;
        rewrite ^/([a-z]+)?-?sitemap\.xsl$ /index.php?xsl=$1 last;
        rewrite ^/sitemap_index\.xml$ /index.php?sitemap=1 last;
        rewrite ^/([^/]+?)-sitemap([0-9]+)?\.xml$ /index.php?sitemap=$1&sitemap_n=$2 last;
        rewrite ^/news-sitemap\.xml$ /index.php?sitemap=wpseo_news last;
        rewrite ^/locations\.kml$ /index.php?sitemap=wpseo_local_kml last;
        rewrite ^/geo-sitemap\.xml$ /index.php?sitemap=wpseo_local last;
        rewrite ^/video-sitemap\.xsl$ /index.php?xsl=video last;
    }
    rewrite ^/shop/minify/([0-9]+)(/.*.(js|css))$ /shop/lib/minify/m.php?f=$2&d=$1 last;
    rewrite ^/shop/skin/m/([0-9]+)(/.*.(js|css))$ /shop/lib/minify/m.php?f=$2&d=$1 last;
    location /shop/lib/minify/ {
      allow all;
    }
    location /shop/skin/m/ {
        rewrite ^/shop/skin/m/([^/]+)(/.*.(js|css))$ /lib/minify/m.php?f=$2&d=$1;
    }
    location  /push-to-deploy {
      rewrite /push-to-deploy(.*) /$1  break;
      proxy_pass         http://localhost:3000;
      proxy_redirect     off;
      proxy_set_header   Host $server_name;
    }
}

[oschaaf]

@dannymichel You have a LoadFromFileRuleMatch which seems to have an unescaped $-sign.

Because Nginx uses the $-sign to indicate script variables, when you turn on ProcessScriptVariables you need to make a small change to any script-supporting (pagespeed) commands that are using $.

For example, if you had:

pagespeed LoadFromFileRuleMatch Disallow \.ssp.css$;

Then you would need to replace $ with $ps_dollar:

pagespeed LoadFromFileRuleMatch Disallow \.ssp.css$ps_dollar;

See https://modpagespeed.com/doc/system#nginx_script_variables for more details.

[dannymichel]

Thank you

[dannymichel]

Does the rest of my config look ok? I really feel like i have a bunch of redundancies.

[oschaaf]

Closing this issue, but feel free to re-open if there does turn out to be an issue with this feature.

[oschaaf]

@dannymichel If you have any specific questions about your configuration, those are probably best asked at the discussion list: https://groups.google.com/forum/#!forum/ngx-pagespeed-discuss

If you do post over there, it would probably be helpful if you can give a bit more context on how you came to that configuration and what you are trying to achieve (or what problem you are running into).