search

nigoroll / libvmod-dynamic

The Varnish dns/named director continued
BSD 2-Clause "Simplified" License
96 stars 34 forks source link

IPv6 backends remain sick - Varnish 7.4.3, vmod_dynamic 2.8.0 #119

Closed gerhard closed 2 months ago

gerhard commented 2 months ago

Given the following DNS records:

dig changelog-2024-01-12.internal AAAA +short
fdaa:0:4556:a7b:303:5118:aa8a:2
fdaa:0:4556:a7b:303:ccc2:4a66:2

And the following dynamic backend config:

backend default none;

probe changelog_health {
  .url = "/health";
  .interval = 5s;
  .timeout = 2s;
  .window = 10;
  .threshold = 5;
}

# Define IPv6 only ACL for the module
acl ipv6_only { "::0"/0; }

# Setup a dynamic director
sub vcl_init {
  new changelog = dynamic.director(
    ttl = 1m,
    probe = changelog_health,
    first_byte_timeout = 5s,
    connect_timeout = 5s,
    between_bytes_timeout = 30s,
    whitelist = ipv6_only
  );
}

sub vcl_recv {
  set req.backend_hint = changelog.backend("changelog-2024-01-12.internal", "4000");
}

Even though the backends are resolved correctly, they remain sick:

varnishadm backend.list
Backend name                                         Admin  Probe  Health  Last change
boot.changelog(changelog-2024-01-12.internal:4000)   probe  0/2    sick    Mon, 19 Aug 2024 07:17:07 GMT
boot.changelog(fdaa:0:4556:a7b:303:5118:aa8a:2:4000) probe  0/10   sick    Mon, 19 Aug 2024 07:17:07 GMT
boot.changelog(fdaa:0:4556:a7b:303:ccc2:4a66:2:4000) probe  0/10   sick    Mon, 19 Aug 2024 07:17:07 GMT

This is what I see in varnishlog:

varnishlog -g raw -i backend_health
         0 Backend_health - changelog(fdaa:0:4556:a7b:303:ccc2:4a66:2:4000) Still sick -6--X-R- 0 5 10 0.001638 0.000000 "HTTP/1.1 400 Bad Request"
         0 Backend_health - changelog(fdaa:0:4556:a7b:303:5118:aa8a:2:4000) Still sick -6--X-R- 0 5 10 0.001533 0.000000 "HTTP/1.1 400 Bad Request"
         0 Backend_health - changelog(fdaa:0:4556:a7b:303:5118:aa8a:2:4000) Still sick -6--X-R- 0 5 10 0.001314 0.000000 "HTTP/1.1 400 Bad Request"
         0 Backend_health - changelog(fdaa:0:4556:a7b:303:ccc2:4a66:2:4000) Still sick -6--X-R- 0 5 10 0.001499 0.000000 "HTTP/1.1 400 Bad Request"
...

And yet if I use curl, both these backends work:

curl -v6 http://[fdaa:0:4556:a7b:303:ccc2:4a66:2]:4000/health
*   Trying [fdaa:0:4556:a7b:303:ccc2:4a66:2]:4000...
* Connected to fdaa:0:4556:a7b:303:ccc2:4a66:2 (fdaa:0:4556:a7b:303:ccc2:4a66:2) port 4000 (#0)
> GET /health HTTP/1.1
> Host: [fdaa:0:4556:a7b:303:ccc2:4a66:2]:4000
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 200 OK
< cache-control: max-age=0, private, must-revalidate
< content-length: 0
< date: Mon, 19 Aug 2024 07:28:15 GMT
< server: Cowboy
<
* Connection #0 to host fdaa:0:4556:a7b:303:ccc2:4a66:2 left intact

curl -v6 http://[fdaa:0:4556:a7b:303:5118:aa8a:2]:4000/health
*   Trying [fdaa:0:4556:a7b:303:5118:aa8a:2]:4000...
* Connected to fdaa:0:4556:a7b:303:5118:aa8a:2 (fdaa:0:4556:a7b:303:5118:aa8a:2) port 4000 (#0)
> GET /health HTTP/1.1
> Host: [fdaa:0:4556:a7b:303:5118:aa8a:2]:4000
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 200 OK
< cache-control: max-age=0, private, must-revalidate
< content-length: 0
< date: Mon, 19 Aug 2024 07:28:48 GMT
< server: Cowboy
<
* Connection #0 to host fdaa:0:4556:a7b:303:5118:aa8a:2 left intact

I think that the backend IPv6 is not interpolated correctly - changelog(fdaa:0:4556:a7b:303:ccc2:4a66:2:4000) should be changelog([fdaa:0:4556:a7b:303:ccc2:4a66:2]:4000) but maybe the formatting is misleading me.

Is there anything that I could do to debug this further?

FTR:

gerhard commented 2 months ago

My issue was that I was missing the host_header property in dynamic.director

This is the dynamic director definition that works:

sub vcl_init {
  new changelog = dynamic.director(
    ttl = 10s,
    probe = changelog_health,
    host_header = "changelog-2024-01-12.fly.dev",
    first_byte_timeout = 5s,
    connect_timeout = 5s,
    between_bytes_timeout = 30s,
    whitelist = ipv6_only
  );
}