saintmode.blacklist problem with returning abandon (with ESI resources)

[JereMalinen]

Hey,

I'm doing migration from Varnish 3 to 5. If ESI resource fails, we want to return empty synthetic response. Returning empty response works, but Varnish 5.2 makes backend request every time even though saintmode.blacklist should be active (i.e. if some ESI url went broke, try again after a minute, not hundreds of times per second).

In 3.x it was working:

sub vcl_fetch {
   if (req.esi_level > 0 && beresp.status >= 400) {
      set beresp.saintmode = 60s;
      error 351;
   }
  ...
}

sub vcl_error {
        # ESI error (351 only on first request, then 503 due to "saintmode")
        if (obj.status == 351 || (obj.status == 503 && req.url ~ "^/esi/.*")) {
           synthetic {""};
           return (deliver);
        }
        ...
}

5.x vcl snippets (saintmode.blacklist(60s); doesn't affect):

sub vcl_backend_response {
     if (beresp.status >= 400) {
       saintmode.blacklist(60s);
       return (abandon);
     }
     ...
}

sub vcl_synth {
  if (resp.status == 503 && req.url ~ "^/esi/.*") {
        synthetic("");
        return(deliver);
  }
  ...
}

So am I missing some relevant piece of information? Or has the feature changed?

[dmatetelki]

Hi @JereMalinen,

The following test-VCL worked for me with VC-5.2. Can it be that you forgot setting up the saintmode VMOD in vcl_init and vcl_backend_fetch?

The syntax changed quite a bit since VC-3, I followed the instructions at: https://github.com/varnish/varnish-modules/blob/master/docs/vmod_saintmode.rst

vcl 4.0;

import saintmode;
import directors;

backend default {
        .host = "127.0.0.1";
        .port = "80";
}

sub vcl_init {
        new sm1 = saintmode.saintmode(default, 10);

        new imagedirector = directors.random();
        imagedirector.add_backend(sm1.backend(), 1);
}

sub vcl_backend_fetch {
        set bereq.backend = imagedirector.backend();
}

sub vcl_backend_response {

        set beresp.do_esi = true;

        if (beresp.status >= 400) {
                saintmode.blacklist(60s);
                return (abandon);
        }
}

# first time
sub vcl_synth {
        if (resp.status == 503 && req.url ~ "^/esi/.*") {
                synthetic("");
                return(deliver);
        }
}

# later
sub vcl_backend_error {
        if (beresp.status == 503 && bereq.url ~ "^/esi/.*") {
                synthetic("");
                return(deliver);
        }
}

[JereMalinen]

Thanks for response. I think I tried setting up saintmode, but we don't wish to blacklist whole backend in any case (i.e. single resource might be broken on all servers and it isn't server's fault), so I think I tried new web_sm = saintmode.saintmode(web, 0); if I remember correctly and then blacklisting single ESI-urls doesn't work.

Is using enough big threshold viable solution, i.e. new web_sm = saintmode.saintmode(web, 9999999999999);?

[gquintard]

Hi,

Using a big threshold is indeed a solution.

However, this space is used only to track bugs, so please continue this discussion using one of the help channels: http://varnish-cache.org/support/index.html