Does not build against varnish 5.0.0

[ssm]

The last tag 0.9.1, and the current master (b8c0060b51b0130960e3fb8b67c3a30d26cfe296) does not build against Varnish 5.0.0.

Originally reported to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=838114

vmod_xkey fails with:

vmod_xkey.c:456:10: error: 'enum exp_event_e' declared inside parameter list will not be visible outside of this definition or declaration [-Werror]
     enum exp_event_e event, void *priv)
          ^~~~~~~~~~~
vmod_xkey.c:456:22: error: parameter 3 ('event') has incomplete type
     enum exp_event_e event, void *priv)
                      ^~~~~
vmod_xkey.c: In function 'xkey_cb':
vmod_xkey.c:465:7: error: 'EXP_INSERT' undeclared (first use in this function)
  case EXP_INSERT:
       ^~~~~~~~~~
vmod_xkey.c:465:7: note: each undeclared identifier is reported only once for each function it appears in
vmod_xkey.c:466:7: error: 'EXP_INJECT' undeclared (first use in this function)
  case EXP_INJECT:
       ^~~~~~~~~~
vmod_xkey.c:469:7: error: 'EXP_REMOVE' undeclared (first use in this function)
  case EXP_REMOVE:
       ^~~~~~~~~~
vmod_xkey.c:456:22: error: unused parameter 'event' [-Werror=unused-parameter]
     enum exp_event_e event, void *priv)
                      ^~~~~
vmod_xkey.c: In function 'purge':
vmod_xkey.c:511:18: error: 'struct objcore' has no member named 'exp'; did you mean 'keep'?
       oc->objcore->exp.ttl <= (ctx->now - oc->objcore->exp.t_origin))
                  ^~
vmod_xkey.c:511:54: error: 'struct objcore' has no member named 'exp'; did you mean 'keep'?
       oc->objcore->exp.ttl <= (ctx->now - oc->objcore->exp.t_origin))
                                                      ^~
vmod_xkey.c:524:19: error: 'struct objcore' has no member named 'exp'; did you mean 'keep'?
        oc->objcore->exp.grace, oc->objcore->exp.keep);
                   ^~
vmod_xkey.c:524:43: error: 'struct objcore' has no member named 'exp'; did you mean 'keep'?
        oc->objcore->exp.grace, oc->objcore->exp.keep);
                                           ^~
vmod_xkey.c:526:38: error: 'struct objcore' has no member named 'exp'; did you mean 'keep'?
    EXP_Rearm(oc->objcore, oc->objcore->exp.t_origin, 0,
                                      ^~
vmod_xkey.c: In function 'vmod_event':
vmod_xkey.c:559:8: error: implicit declaration of function 'EXP_Register_Callback' [-Werror=implicit-function-declaration]
        EXP_Register_Callback(xkey_cb, NULL);
        ^~~~~~~~~~~~~~~~~~~~~
vmod_xkey.c:572:4: error: implicit declaration of function 'EXP_Deregister_Callback' [-Werror=implicit-function-declaration]
    EXP_Deregister_Callback(&xkey_cb_handle);
    ^~~~~~~~~~~~~~~~~~~~~~~

In addition, vmod_softpurge fails with:

/bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -I/usr/include/varnish -I../src/foreign -Wall -Werror -Wunused-parameter -Wmissing-prototypes   -g -O2 -MT vmod_softpurge.lo -MD -MP -MF .deps/vmod_softpurge.Tpo -c -o vmod_softpurge.lo vmod_softpurge.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I/usr/include/varnish -I../src/foreign -Wall -Werror -Wunused-parameter -Wmissing-prototypes -g -O2 -MT vmod_softpurge.lo -MD -MP -MF .deps/vmod_softpurge.Tpo -c vmod_softpurge.c  -fPIC -DPIC -o .libs/vmod_softpurge.o
In file included from /usr/include/varnish/common/common.h:39:0,
                 from /usr/include/varnish/cache/cache.h:39,
                 from vmod_softpurge.c:30:
vmod_softpurge.c: In function ‘vmod_softpurge’:
/usr/include/varnish/vqueue.h:540:39: error: ‘struct objcore’ has no member named ‘list’
 #define VTAILQ_NEXT(elm, field) ((elm)->field.vtqe_next)
                                       ^
/usr/include/varnish/vqueue.h:460:14: note: in expansion of macro ‘VTAILQ_NEXT’
      (var) = VTAILQ_NEXT((var), field))
              ^~~~~~~~~~~
vmod_softpurge.c:63:2: note: in expansion of macro ‘VTAILQ_FOREACH’
  VTAILQ_FOREACH(oc, &oh->objcs, list) {
  ^~~~~~~~~~~~~~
vmod_softpurge.c:68:23: error: ‘OC_EF_DYING’ undeclared (first use in this function)
   if (oc->exp_flags & OC_EF_DYING)
                       ^~~~~~~~~~~
vmod_softpurge.c:68:23: note: each undeclared identifier is reported only once for each function it appears in
vmod_softpurge.c:85:27: error: ‘struct objcore’ has no member named ‘exp’; did you mean ‘keep’?
   EXP_Rearm(oc, now, 0, oc->exp.grace, oc->exp.keep);
                           ^~
vmod_softpurge.c:85:42: error: ‘struct objcore’ has no member named ‘exp’; did you mean ‘keep’?
   EXP_Rearm(oc, now, 0, oc->exp.grace, oc->exp.keep);
                                          ^~

I guess varnishcache/varnish-cache@4ded44b6c9633d483de1609a97c99fa0b6db153a, is one of the related changes in Varnish Cache.

[aondio]

Hi @ssm, thanks for the report. I haven't tested this, but I believe this collection hasn't been adjusted to varnish 5.0 yet.

I'll get back to you when I know more.

[dridi]

Quick glance:

Code was refactored, breaking things hard. Good thing we bumped the VRT ABI major just in case.

The EXP_* symbols are now called OC_EF_* and belong in (struct objcore).exp_flags.

I think that (struct objcore).list was renamed to hsh_list.

OC_EF_DYING is now in (struct objcore).flags: OC_F_DYING.

No time to look at the rest, may resume later.

[dridi]

@ssm please find attached a patch that should be enough for your debian build. Tests pass on my Fedora box.

debian.patch.txt

Others: this patch doesn't deal with Varnish Plus differences.

[ssm]

@Dridi Thanks, adding that patch to the Debian package worked fine.

A new version of varnish-modules has been uploaded to Debian unstable (https://tracker.debian.org/news/799024).

[dridi]

@ssm thanks for confirming.

@aondio you can assign it to me if you want, I'll produce a sustainable fix.

[aondio]

@Dridi, it's all yours :)

[dridi]

Reopening to point out that everything should work on Varnish Plus except the use of resp.msg in test cases.

See varnishcache/varnish-cache#2097 for a back-portable fix.

[dridi]

Oh, and it breaks on Travis CI because Varnish 4.1 is installed, but we don't have packages for Varnish 5 yet. Please let me know how to proceed with Travis, I can build Varnish from source or wait until a 5.0 repo pops up.

[nigoroll]

Hey folks, you're doing duplicate work. I've been maintaining a master version of this repo for quite some time: https://github.com/nigoroll/varnish-modules It's even mentioned in the readme

[dridi]

FYI I somehow just remembered that @nigoroll has a fork building against varnish-master.

[fgsch]

I've just committed some changes that should cover 4.1 and master as of today, and hopefully also 5.0. @ssm can you give it a try and let me know if anything else needs attention?

[fgsch]

I can confirm that this also fixes the build against 5.0.0. For the records, this has been compiled and tested with the follow branches/repositories:

varnish-4.1
varnish-4.1.3
varnish-4.1-plus
varnish-5.0.0
varnish-master

[fgsch]

Fixed in 29923f976bb737a81a855a13b0acbdcac26440d5.