Object resolver not initialized?

cosimo commented 5 months ago

Hi!

I'm trying to use libvmod_dynamic with varnish 7.4:

varnishcache/varnish-cache@b659b7ae62b44c05888919c5c1cd03ba6eaec681
libvmod-dynamic rev: 3697d6f195fe077fe213918b7b67f5da4efdede2

This is on Ubuntu 20.04, and I've build libvmod_dynamic.so using this Dockerfile: https://gist.github.com/cosimo/314e441313426a93bdfd3438059e44f9

My runtime environment is varnish 7.4.3-1~focal on Ubuntu 20.04.6 TLS x86_64 from the https://packagecloud.io/varnishcache/varnish74/ubuntu/ focal main debian repository.

The relevant VCL code is:

vcl 4.0;
import dynamic;

backend default none;

sub vcl_init {
    new r = dynamic.resolver();
    new rest = dynamic.director(
      port = "80",
      resolver = r.use(),
      ttl_from = dns
    );
}

sub vcl_recv {
    # `_rest._tcp.example.com` is an SRV record
    set req.backend_hint = rest.backend("_rest._tcp.example.com");
}

When starting up varnish, I'm getting the following error:

Mar 25 14:27:35 host varnishd[2689915]: Version: varnish-7.4.3 revision b659b7ae62b44c05888919c5c1cd03ba6eaec681
Mar 25 14:27:35 host varnishd[2689915]: Platform: Linux,5.15.0-1053-aws,x86_64,-junix,-smalloc,-sdefault,-hcritbit
Mar 25 14:27:35 host varnishd[2689915]: Child (2689931) Started
Mar 25 14:27:35 host varnishd[2689915]: Child launched OK
Mar 25 14:27:35 host varnishd[2689915]: Error: Child (2689931) Pushing vcls failed:
Mar 25 14:27:35 host varnishd[2689915]: VCL "boot" Failed initialization
Mar 25 14:27:35 host varnishd[2689915]: Message:
Mar 25 14:27:35 host varnishd[2689915]:         Object r not initialized
Mar 25 14:27:35 host varnishd[2689915]: Info: Child (2689931) said Child starts
Mar 25 14:27:35 host varnishd[2689915]: Child (2689931) Pushing vcls failed:
                                                                      VCL "boot" Failed initialization
                                                                      Message:
                                                                              Object r not initialized
Mar 25 14:27:35 host varnishd[2689915]: Stopping Child
Mar 25 14:27:35 host varnishd[2689915]: Child (2689931) said Child starts
Mar 25 14:27:35 host varnishd[2689915]: Info: Child (2689931) said Child dies
Mar 25 14:27:35 host varnishd[2689915]: Info: Child (2689931) ended
Mar 25 14:27:35 host varnishd[2689915]: Child (2689931) said Child dies
Mar 25 14:27:35 host varnishd[2689915]: Child (2689931) ended
Mar 25 14:27:35 host varnishd[2689915]: Child cleanup complete

Have looked at the documentation in the vmod_dynamic.vcc but I haven't really understood the reason why the resolver object is not initialized.

Other things I've noticed:

When querying varnishadm backend.list -j I see that the backends seem to be considered sick, as in:

root@host:/etc/varnish# varnishadm backend.list -j
[ 3, ["backend.list", "-j"], 1711378545.538,
{
"boot.rest(_rest._tcp.example.com:(null))": {
  "type": "dynamic",
  "admin_health": "probe",
  "probe_message": [0, 0, "sick"],
  "last_change": 1711378480.950
}
}
]

I can't seem to be able to use the ttl_from = dns option, the error message mentioned that I need to use a resolver, which I am indeed doing.
When running make check from inside the Docker container build, 1 test is failing. Here's the output I'm seeing:

PASS: tests/layer.vtc
PASS: tests/r00107.vtc
PASS: tests/stale_obj.vtc
PASS: tests/test01.vtc
PASS: tests/test02.vtc
PASS: tests/test03.vtc
PASS: tests/test04.vtc
PASS: tests/test05.vtc
PASS: tests/test06.vtc
PASS: tests/test07.vtc
PASS: tests/test08.vtc
PASS: tests/test09.vtc
PASS: tests/test10.vtc
PASS: tests/test11.vtc
PASS: tests/test12.vtc
FAIL: tests/test13.vtc
PASS: tests/test14.vtc
PASS: tests/via.vtc

Would you have any guidance, or examples of usage of libvmod-dynamic? I haven't found much doing web searches or checking other repositories on github.

Thanks!

nigoroll commented 5 months ago

Your dockerfile does not show getdns being installed, which is required for resolver objects, see README.rst. I agree that the error message could be more helpful, though. The vcc file gets translated into RST which, in turn, gets installed as the man page. It is intended as the reference documentation. If you are missing anything I would appreciate contributions to it or, alternatively, more concrete questions which are currently unanswered. For usage examples, you might also want to look at the test cases for resolver and service.

cosimo commented 5 months ago

Thanks @nigoroll for the pointers.

The docker build doesn't have libgetdns, but the ubuntu 20.04 runtime environment has the libgetdns10 package installed. I now included the lib in the docker build. I expected the .so file to use libgetdns but I don't see that:

root@75da4023b00c:/# ldd /tmp/libvmod-dynamic/src/.libs/libvmod_dynamic.so
        linux-vdso.so.1 (0x00007fff565a2000)
        libvarnishapi.so.3 => /lib/libvarnishapi.so.3 (0x00007fccbcea0000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fccbce7d000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fccbcc8b000)
        libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007fccbcbfa000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fccbcaab000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fccbcef9000)

Will investigate a bit more.

UPDATE: I probably need libgetdns-dev to be installed in the docker container...

nigoroll commented 5 months ago

Did you install the -dev package as documented?

UPDATE: crossed with the update to the previous message.

cosimo commented 5 months ago

I managed to get past the Object resolver not initialized using the libvmod_dynamic.so built correctly with libgetdns.

Still can't get the SRV record lookup working correctly... I must be doing something wrong. Closing this at least, the original problem is solved.

I updated the Dockerfile so it builds a correct libvmod_dynamic.so at least. That might be useful to others.

nigoroll commented 5 months ago

This needs an error message improvement. Regarding your SRV record lookup, do you use .service()? .backend() in your example is wrong, I thought you might have tried it out of desperation that .service() did not work, so I did not point this out separately.

cosimo commented 5 months ago

Hey @nigoroll, I did just a few minutes ago manage to get the SRV lookup to work, I used the ./src/tests/service/*.vtc files as reference, and there I saw I should have used the .service() function.

I will test this a bit now, trying out the different parameters, etc...

nigoroll / libvmod-dynamic

Object resolver not initialized? #114