can't extract distfiles with EXTRACT_SUFX=.tar.lz

[schmonz]

devel/libidn2 2.0.3 is packaged from a .tar.lz distfile.

As a one-off build directly from pkgsrc, it extracts fine. I've also been watching pkgsrc-bulk@ and not seeing any reported failures. That's why I suspect the following must be a bug in pkg_comp:

With my usual weekly build, I get the following in configure.log:

=> Bootstrap dependency digest>=20010302: found digest-20160304
=> Checksum SHA1 OK for libidn2-2.0.3.tar.lz
=> Checksum RMD160 OK for libidn2-2.0.3.tar.lz
=> Checksum SHA512 OK for libidn2-2.0.3.tar.lz
===> Installing dependencies for libidn2-2.0.3
=> Tool dependency libtool-base>=2.4.2nb9: found libtool-base-2.4.6
=> Tool dependency lzip>=1.14: found lzip-1.19
=> Tool dependency pkgconf-[0-9]*: found pkgconf-1.3.5
=> Tool dependency checkperms>=1.1: found checkperms-1.11nb1
=> Build dependency cwrappers>=20150314: found cwrappers-20170611
=> Full dependency libunistring>=0.9.4: found libunistring-0.9.7
===> Overriding tools for libidn2-2.0.3
===> Extracting for libidn2-2.0.3
lzip: not found
tar: End of archive volume 1 reached
tar: Sorry, unable to determine archive format.
*** Error code 1

Stop.
bmake: stopped in /pkg_comp/pkgsrc/devel/libidn2
*** Error code 1

Stop.
bmake: stopped in /pkg_comp/pkgsrc/devel/libidn2

I can reproduce just as well with a simple pkg_comp auto libidn2 (though when I do it this way, the build logs disappear before I can save them off!).

[schmonz]

Just encountered this again, with wget-1.19.2.

[schmonz]

EXTRACT_SUFX=.tar.xz is becoming more and more prevalent in pkgsrc, making it less and less likely that pkg_comp will succeed at building any given set of packages. My own weekly build has been failing for the last few weeks.

[schmonz]

Whoops (2x):

1. I meant .tar.lz
2. Just did another -current build and it succeeded

It's probably still true that the situation is tending toward worse, but maybe it's not as urgently bad yet.

[jmmv]

Sorry, I haven't had a chance to look at this. Do you have any hints as to what may be happening? On which platform are you seeing this?

[schmonz]

This is on NetBSD/amd64 7.1. I haven't come up with any guesses what's going on. Here are my configs, in case they have anything to do with it:

• pkgcomp-conf
• pkgcomp-autopackages
• pkgcomp-extramkconf
• pkgcomp-sandboxconf

[jmmv]

Apologies for the really long delay... do you know if this is still happening for you?

I just gave this a try by enabling .tar.lz on libidn2... and couldn't reproduce on 8.0 amd64: lzip is correctly installed as a dependency and later used in the build.

If I had to guess, I suspect this may be caused by your custom setting of PKGSRC_SETENV, but I don't know why that would be at this point -- and I cannot reproduce the failure by setting this variable.

[schmonz]

It's been a while since I noticed myself having this problem. Very few packages are using .tar.lz, and my AUTO_PACKAGES list doesn't include any of them.

I'm also on 8.0 amd64 now. With a clean and up-to-date cvs checkout of pkgsrc-current, with devel/libidn2 twiddled to use a .tar.lz distfile, running pkg_comp auto libidn2 gives this:

[1/9] Starting build of  cwrappers-20180325
[1/9] Successfully built cwrappers-20180325
[2/9] Starting build of  digest-20160304
[2/9] Successfully built digest-20160304
[3/9] Starting build of  checkperms-1.12
[3/9] Successfully built checkperms-1.12
[4/9] Starting build of  m4-1.4.18
[4/9] Successfully built m4-1.4.18
[5/9] Starting build of  libtool-base-2.4.6nb1
[5/9] Successfully built libtool-base-2.4.6nb1
[6/9] Starting build of  lzip-1.20
[6/9] Successfully built lzip-1.20
[7/9] Starting build of  libunistring-0.9.10
[7/9] Successfully built libunistring-0.9.10
[8/9] Starting build of  pkgconf-1.4.1nb1
[8/9] Successfully built pkgconf-1.4.1nb1
[9/9] Starting build of  libidn2-2.0.5
[9/9] Failed to build    libidn2-2.0.5

How can I provide more information?

[schmonz]

Still happening on NetBSD/amd64 8.1, with wget:

===> Extracting for wget-1.20.3nb4
/pkg_comp/pkgsrc/mk/extract/extract: lzip: not found

[schmonz]

If I had to guess, I suspect this may be caused by your custom setting of PKGSRC_SETENV, but I don't know why that would be at this point -- and I cannot reproduce the failure by setting this variable.

Trying a build with AUTO_PACKAGES pared down to just www/libpsl (which comes as .tar.lz):

• If I keep my custom PKGSRC_SETENV, it fails to extract as above
• If I comment that line out, it extracts and builds fine

I don't remember whether I tried this when you suggested it 1.5 years ago. Anyway, it looks likely to be the difference-maker. Will report back (and probably close this issue) after a full build has completed.