[Task] Publish v2.32.0.

I'd like to publish a new release. Are there any last "minute" fixes that need to be done? The new wml seems to render the quad-pres based talks over at https://www.shlomifish.org/lecture/ as well as before but we may need some more thorough testing.

@xtaran : I'm assigning it to you; if you think we're good to go, feel free to resolve this issue or unassign yourself. Thanks in advance.

First impression: Did we get new build dependencies? cmake . runs through, but a following make fails as follows:

[ 36%] Linking C executable wml_p3_eperl
/usr/bin/cc -rdynamic -Wl,-E  -fstack-protector-strong -L/usr/local/lib -L/usr/lib/x86_64-linux-gnu/perl/5.30/CORE CMakeFiles/eperl.dir/__/__/lib/getopt.c.o CMakeFiles/eperl.dir/__/__/lib/getopt1.c.o CMakeFiles/eperl.dir/eperl_license.c.o CMakeFiles/eperl.dir/eperl_logo.c.o CMakeFiles/eperl.dir/eperl_readme.c.o CMakeFiles/eperl.dir/eperl_debug.c.o CMakeFiles/eperl.dir/eperl_http.c.o CMakeFiles/eperl.dir/eperl_main.c.o CMakeFiles/eperl.dir/eperl_parse.c.o CMakeFiles/eperl.dir/eperl_perl5.c.o CMakeFiles/eperl.dir/eperl_pp.c.o CMakeFiles/eperl.dir/eperl_sys.c.o -o wml_p3_eperl  -lperl -lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt 
/usr/bin/ld: cannot find -lgdbm_compat
collect2: error: ld returned 1 exit status
make[6]: *** [wml_backend/p3_eperl/CMakeFiles/eperl.dir/build.make:285: wml_backend/p3_eperl/wml_p3_eperl] Error 1
make[5]: *** [CMakeFiles/Makefile2:1196: wml_backend/p3_eperl/CMakeFiles/eperl.dir/all] Error 2
make[4]: *** [Makefile:182: all] Error 2
Running [mkdir /home/abe/wml/website-meta-language/wml_test/FOO && cd /home/abe/wml/website-meta-language/wml_test/FOO && cmake  -D CMAKE_INSTALL_PREFIX=/home/abe/wml/website-meta-language/wml_test/tests/installation /home/abe/wml/wml-upstream/src && make VERBOSE=1 && make package_source && make install] failed! at /home/abe/wml/wml-upstream/src/wml_test/run_test.pl line 16.
make[3]: *** [wml_test/CMakeFiles/check.dir/build.make:76: wml_test/CMakeFiles/check] Error 2
make[2]: *** [CMakeFiles/Makefile2:1763: wml_test/CMakeFiles/check.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:1770: wml_test/CMakeFiles/check.dir/rule] Error 2
make: *** [Makefile:775: check] Error 2

Two things made me wonder here:

Shouldn't that already have failed during the cmake call? (Which as far as I understand is the equivalent to Autotool's configure call.)
Neither Debian's wml source package (built from the same source) nor Debian's eperl source package (built from the original source) so far have libgdbm-compat-dev as build-depdency.
- In case this is a recent feature/performance optimization and not just a recently introduced bug in the CMake configuration of WML: It seems as if this has been added either very recently or in the past few years. But why has been the "GNU dbm legacy support" added and not current GNU dbm support?

Tried to bisect it, but

2.12.2 (WML version in Debian 10, trying to build from upstream source) to 2.22.0 fails due to
```
Unknown CMake command "SHLOMIF_COMMON_SETUP"
```
(I think I've fixed this in Debian by adding that file as a patch or so.)

2.24.0 to 2.26.0 fails with

/usr/bin/ld: CMakeFiles/iselect.dir/iselect_browse.c.o:(.bss+0x0): multiple definition of `Line'; CMakeFiles/iselect.dir/iselect_main.c.o:(.bss+0x0): first defined here`.

And from 2.28.0 onwards it fails with the above mentioned
```
/usr/bin/ld: cannot find -lgdbm_compat
```

Reason why I didn't ran into the latter two in Debian so far: I so far only build the subprojects wml and slice from the wml source package. iselect and eperl are built from their own (original) sources.

Hi @xtaran !

First impression: Did we get new build dependencies? cmake . runs through, but a following make fails as follows:

[ 36%] Linking C executable wml_p3_eperl
/usr/bin/cc -rdynamic -Wl,-E  -fstack-protector-strong -L/usr/local/lib -L/usr/lib/x86_64-linux-gnu/perl/5.30/CORE CMakeFiles/eperl.dir/__/__/lib/getopt.c.o CMakeFiles/eperl.dir/__/__/lib/getopt1.c.o CMakeFiles/eperl.dir/eperl_license.c.o CMakeFiles/eperl.dir/eperl_logo.c.o CMakeFiles/eperl.dir/eperl_readme.c.o CMakeFiles/eperl.dir/eperl_debug.c.o CMakeFiles/eperl.dir/eperl_http.c.o CMakeFiles/eperl.dir/eperl_main.c.o CMakeFiles/eperl.dir/eperl_parse.c.o CMakeFiles/eperl.dir/eperl_perl5.c.o CMakeFiles/eperl.dir/eperl_pp.c.o CMakeFiles/eperl.dir/eperl_sys.c.o -o wml_p3_eperl  -lperl -lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt 
/usr/bin/ld: cannot find -lgdbm_compat
collect2: error: ld returned 1 exit status
make[6]: *** [wml_backend/p3_eperl/CMakeFiles/eperl.dir/build.make:285: wml_backend/p3_eperl/wml_p3_eperl] Error 1
make[5]: *** [CMakeFiles/Makefile2:1196: wml_backend/p3_eperl/CMakeFiles/eperl.dir/all] Error 2
make[4]: *** [Makefile:182: all] Error 2
Running [mkdir /home/abe/wml/website-meta-language/wml_test/FOO && cd /home/abe/wml/website-meta-language/wml_test/FOO && cmake  -D CMAKE_INSTALL_PREFIX=/home/abe/wml/website-meta-language/wml_test/tests/installation /home/abe/wml/wml-upstream/src && make VERBOSE=1 && make package_source && make install] failed! at /home/abe/wml/wml-upstream/src/wml_test/run_test.pl line 16.
make[3]: *** [wml_test/CMakeFiles/check.dir/build.make:76: wml_test/CMakeFiles/check] Error 2
make[2]: *** [CMakeFiles/Makefile2:1763: wml_test/CMakeFiles/check.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:1770: wml_test/CMakeFiles/check.dir/rule] Error 2
make: *** [Makefile:775: check] Error 2

Two things made me wonder here:

* Shouldn't that already have failed during the `cmake` call? (Which as far as I understand is the equivalent to Autotool's `configure` call.)

In principle it should. In practice gdbm_compat is specified by the libperl build flags, and not on all perl installations. I can try to regex for it in the cmake stage and then to use something like https://cmake.org/cmake/help/latest/command/find_library.html (a kludge I admit).

* Neither Debian's `wml` source package (built from the same source) nor Debian's `eperl` source package (built from the original source) so far have `libgdbm-compat-dev` as build-depdency.

  * In case this is a recent feature/performance optimization and not just a recently introduced bug in the CMake configuration of WML: It seems as if this has been added either very recently or in the past few years. But why has been the "GNU dbm legacy support" added and not current GNU dbm support?

I think I got a runtime error with the noncompat gdbm, but I'll investigate.

Tried to bisect it, but

* 2.12.2 (WML version in Debian 10, trying to build from upstream source) to 2.22.0 fails due to
  ```
  Unknown CMake command "SHLOMIF_COMMON_SETUP"
  ```

  (I think I've fixed this in Debian by adding that file as a patch or so.)

Try using an older https://github.com/shlomif/shlomif-cmake-modules commit - in recent versions of it, I removed some cmake macros and functions that were only used in wml to the wml source.

* 2.24.0 to 2.26.0 fails with
  ```
  /usr/bin/ld: CMakeFiles/iselect.dir/iselect_browse.c.o:(.bss+0x0): multiple definition of `Line'; CMakeFiles/iselect.dir/iselect_main.c.o:(.bss+0x0): first defined here`.
  ```

this is caused by recent GCCs stricter handling of non-static global variables declared (and defined) inside includes. I worked around this in a previous bisect by deleting the "add_subdirectory" line and the C sources were properly fixed in recent version.

* And from 2.28.0 onwards it fails with the above mentioned
  ```
  /usr/bin/ld: cannot find -lgdbm_compat
  ```
Reason why I didn't ran into the latter two in Debian so far: I so far only build the subprojects wml and slice from the wml source package. iselect and eperl are built from their own (original) sources.

Thanks for your input.

@xtaran : hi! I think the cmake stage handles the lack of gdbm.* better now. Please check and let me know.

In principle it should. In practice gdbm_compat is specified by the libperl build flags, and not on all perl installations

More or less confirmed: Debian's perl source package has a build-dependency on libgdbm-dev (>= 1.18-3), libgdbm-compat-dev, so it seems to need both.

I initially expected that these two packages are not co-installable, but they actually are. The package conflict declared in ibgdbm-compat-dev is just against an older version of libgdbm-dev.

I think the cmake stage handles the lack of gdbm.* better now.

Ack:

$ cmake src
CMake Error at wml_backend/p3_eperl/CMakeLists.txt:136 (FIND_LIBRARY):
  Could not find found_gdbm using the following names: gdbm_compat

-- Configuring incomplete, errors occurred!
See also "/home/abe/wml/website-meta-language/CMakeFiles/CMakeOutput.log".
See also "/home/abe/wml/website-meta-language/CMakeFiles/CMakeError.log".

I though still don't understand why this is necessary at all.

The -lgdbm_compat clearly comes from somewhere out of the Perl installation since neither the source code of eperl inside this git repo nor Debian's eperl source package contain any gdbm reference. Checked it in both with git grep -i gdbm. But Debian's eperl source package does neither build-depend on libgdbm-dev nor on libgdbm-compat-dev and builds fine without the latter being installed.

So I think the fact that eperl is being linked against gdbm_compat itself is already wrong. Hence I checked Debian's eperl source package closer with regards how it strips that off:

configure.in:perl_libs="`$perl_prog -e 'use Config; print $Config{libs}'`"
configure:perl_libs="`$perl_prog -e 'use Config; print $Config{libs}'`"

Calling this on the system whwere I ran into this, results in the following:

$ perl -e 'use Config; print $Config{libs}'
-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt

This also makes it into the generated Makefile.stand:

$ fgrep -r gdbm Makefile*
Makefile.stand:perl_libs       = -lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt

This is also in the build log, but solely during the configure run:

$ fgrep -i gdbm ../eperl_2.2.14-24_amd64.build
checking for Perl library files... -lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
checking for Perl library files... -lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt

Then again the resulting binary (as packaged as well as just compiled) doesn't seem to be compiled against gdbm_compat, just against gdbm:

$ eperl -V | fgrep -i gdbm
  System Libs     : -lgdbm -ldl -lm -lpthread -lc -lcrypt
$ ./eperl -V | fgrep -i gdbm
  System Libs     : -lgdbm -ldl -lm -lpthread -lc -lcrypt

I grepped further through Debian's eperl source package and also its generated Makefiles, but I so far couldn't find the place where the -lgdbm_compat is actually dropped. (Looking for that as I think that should be done in in the CMake based variant in this git repo, too.) Grepping through the whole directory after an (inplace) build results solely in these matches:

$ fgrep -r gdbm_compat
config.status:S["perl_libs"]="-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt"
Makefile.stand:perl_libs       = -lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
config.log:configure:2328: result: -lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
config.log:perl_libs='-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt'
config_sc.h:#define AC_perl_libs      "-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt"

Grepping just for just gdbm instead results in some more hits inside just compiled binary files:

Binary file eperl_perl5.o matches
Binary file debian/eperl/usr/bin/eperl matches
Binary file eperl matches

(and one irrelevant hit inside the historical file contrib/pkg/dpkg/eperl.diff)

Looking for the actual gcc linker calls in the build log reveals that the value in perl_libs or AC_perl_libs never gets used:

$ fgrep -i "x86_64-linux-gnu-gcc -Wl" ../eperl_2.2.14-24_amd64.build
x86_64-linux-gnu-gcc -Wl,-z,relro -Wl,-z,now -Wl,-E -fstack-protector-strong -L/usr/local/lib -L/usr/lib/x86_64-linux-gnu/perl/5.30/CORE  -o eperl eperl_main.o eperl_perl5.o eperl_parse.o eperl_pp.o eperl_sys.o eperl_http.o eperl_getopt.o eperl_debug.o eperl_config.o eperl_version.o eperl_readme.o eperl_license.o eperl_logo.o eperl_powered.o -Wl,-E  -fstack-protector-strong -L/usr/local/lib  -L/usr/lib/x86_64-linux-gnu/perl/5.30/CORE -lperl -ldl -lm -lpthread -lc -lcrypt -lperl -ldl -lm -lpthread -lc -lcrypt 
x86_64-linux-gnu-gcc -Wl,-z,relro -Wl,-z,now -Wl,-E -fstack-protector-strong -L/usr/local/lib -L/usr/lib/x86_64-linux-gnu/perl/5.30/CORE  -o eperl eperl_main.o eperl_perl5.o eperl_parse.o eperl_pp.o eperl_sys.o eperl_http.o eperl_getopt.o eperl_debug.o eperl_config.o eperl_version.o eperl_readme.o eperl_license.o eperl_logo.o eperl_powered.o -Wl,-E  -fstack-protector-strong -L/usr/local/lib  -L/usr/lib/x86_64-linux-gnu/perl/5.30/CORE -lperl -ldl -lm -lpthread -lc -lcrypt -lperl -ldl -lm -lpthread -lc -lcrypt

Could this be an issue with the order of parameters passed to the C compiler? IIRC it got picky about that in the past few years. In that case it might be a bug in the current eperl package in Debian Unstable. But then again: That binary works as expected.

Sorry, currently don't have more time to dig deeper here. I think for now the current variant is ok-ish as it as least tells what is wrong.

In case you want to have a look at that, too, it's the git repository at https://salsa.debian.org/debian/eperl — patches can be applied with env QUILT_PATCHES=debian/patches quilt push -a.

@xtaran : thanks for the research and detailed analysis. I see that https://salsa.debian.org/debian/eperl is still using GNU autoconf whereas the eperl bundled in the thewml repo was converted to cmake (and I've likely applied many other changes). I'll try to get to publishing v2.32.0 soon.

I see that https://salsa.debian.org/debian/eperl is still using GNU autoconf

Of course! (SCNR)

whereas the eperl bundled in the thewml repo was converted to cmake

Yeah, and my gut feeling is that either one is wrong with this regards. I'm just not sure which one. But since the cmake variant bails out on issues which don't seem to be present or at least don't harm the autoconf one…

(and I've likely applied many other changes).

Yeah, I tried to diff it, but no changes seem to have been related to that issue on a first glance.

I'll try to get to publishing v2.32.0 soon.

Ok, thanks.

I see that https://salsa.debian.org/debian/eperl is still using GNU autoconf

Of course! (SCNR)

https://en.wiktionary.org/wiki/SCNR . It is not a secret that I dislike GNU Autotools, but I was just noting it as a fact. 🙂

whereas the eperl bundled in the thewml repo was converted to cmake

Yeah, and my gut feeling is that either one is wrong with this regards. I'm just not sure which one. But since the cmake variant bails out on issues which don't seem to be present or at least don't harm the autoconf one…

Yes, you may be right. We can try regexing out the extraneous -lgdbm.* flags, but they may be needed elsewhere, and AFAIK should be mostly harmless if given. The only harm I can think of is a (probably minorly) higher start up time and RAM consumption.

(and I've likely applied many other changes).

Yeah, I tried to diff it, but no changes seem to have been related to that issue on a first glance.

I think you're right. Most of these changes were cleanups, optimisations, some fixes for getting it to run on cygwin, and removing workarounds for older Unix-like systems and pre-C99/etc. compilers.

I'll try to get to publishing v2.32.0 soon.

Ok, thanks.

Done now: https://github.com/thewml/website-meta-language/releases/tag/releases%2Fwml-2.32.0 . It seems to have been built fine on http://pkgsubmit.mageia.org/ as well. Thanks! Resolving.

thewml / website-meta-language

[Task] Publish v2.32.0. #44