search

StrawberryPerl / build-extlibs

16 stars 11 forks source link

Summary of whats blocking perl builds? #4

Closed djzort closed 9 months ago

djzort commented 1 year ago

I have from tpf slack the following irc snippet https://www.irccloud.com/pastebin/Bt8wnGHH/explanation%20of%20current%20problem.txt

Which indicates that reliance on Z: needs to be reworked for builds to continue in Docker?

I am hoping someone can provide a summary of whats blocking strawberry perl builds.

shawnlaffan commented 1 year ago

To what extent can the external libs be built using Aliens? These handle patching and the like and are not dependent on drive paths.

It should be possible to sequentially build the respective Aliens as shared installs, collating the folder structures along the way into what will become the strawberry/c dir so subsequent aliens pick up their dependencies. Appending the trailing underscores will in most cases be a rename. Or perhaps there is a means within Alien::Build to do so.

And can some of the lowest level dependencies be first copied across from msys2 and renamed? e.g. libtermcap has no non-system dependencies so perhaps it could be renamed with the trailing underscores, and any pkgconfig data updated as needed.

One caveat is that not all Aliens support dynamic builds, but this could be a matter of updates to the respective Aliens.

genio commented 1 year ago

The Z: drive issue has been worked around and corrected to work in Docker. We have run through the build process and have a few errors to work through: https://gist.github.com/genio/655942d9b9b9fd6ec406c3ccfccfcafc

shawnlaffan commented 1 year ago

Thanks. That's good to hear.

genio commented 1 year ago

You can check out the repo with the dockerfile and run the commands yourself if you'd like to help debug: https://github.com/StrawberryPerl/spbuild#synopsis

djzort commented 1 year ago

Would it be possible to create issues for each broken thing, so that people can chip away at them?

genio commented 1 year ago

Check out the latest spbuild repo, create your own fork of the build-extlibs repo and create a build script named 5034-small or some such. In that script, only add the first two libraries:

termcap-1.3.1
readline-8.1

Then run through the build process lined out in https://github.com/StrawberryPerl/spbuild#synopsis against that small file. You'll see that readline fails. If you can figure out why that's failing, please let me know.

genio commented 1 year ago

Then, rinse and repeat for the other failures in my gist above

shawnlaffan commented 1 year ago

Looking at the more detailed build logs now I'm running it, the readline error looks to have been reported upstream with a patch: https://lists.gnu.org/archive/html/bug-readline/2021-02/msg00007.html

Edit: The patch appears to be in readline 8.2, released 2022-09-26

genio commented 1 year ago

Great news. Thank you for looking into that. I'll try to kick off a build this evening with 8.2 instead of 8.1 and then check into the next thing that fails.

genio commented 1 year ago 
ContainerAdministrator@95fd81efafc8 MINGW64 /z/extlib
# ./build.sh 5034-small __
###### [12:40:00] BUILD STARTED param1='5034-small' param2='__'
...trimmed...
######## retval=success
######## gonna launch: make
test -d shlib || mkdir shlib
( cd shlib ; make  all )
... trimmed ...
../rlprivate.h:308:37: error: unknown type name 'fd_set'
  308 | extern int _rl_timeout_select (int, fd_set *, fd_set *, fd_set *, const struct timeval *, const sigset_t *);
      |                                     ^~~~~~
../rlprivate.h:308:47: error: unknown type name 'fd_set'
  308 | extern int _rl_timeout_select (int, fd_set *, fd_set *, fd_set *, const struct timeval *, const sigset_t *);
      |                                               ^~~~~~
../rlprivate.h:308:57: error: unknown type name 'fd_set'
  308 | extern int _rl_timeout_select (int, fd_set *, fd_set *, fd_set *, const struct timeval *, const sigset_t *);
      |                                                         ^~~~~~
../rlprivate.h:308:97: error: unknown type name 'sigset_t'
  308 | extern int _rl_timeout_select (int, fd_set *, fd_set *, fd_set *, const struct timeval *, const sigset_t *);
      |                                                                                                 ^~~~~~~~
make[1]: *** [Makefile:116: parens.so] Error 1
make[1]: Leaving directory '/c/spbuild/extlib/_5034-small__.src/readline-8.2/shlib'
make: *** [Makefile:219: shared] Error 2
######## retval=FAILURE
... trimmed ...

ContainerAdministrator@95fd81efafc8 MINGW64 /z/extlib
# grep -E 'retval=' _5034-small__/*.build.log
_5034-small__/readline-8.2.build.log:######## retval=success
_5034-small__/readline-8.2.build.log:######## retval=FAILURE
_5034-small__/readline-8.2.build.log:######## retval=FAILURE
_5034-small__/termcap-1.3.1.build.log:######## retval=success
_5034-small__/termcap-1.3.1.build.log:######## retval=success
_5034-small__/termcap-1.3.1.build.log:######## retval=success
_5034-small__/termcap-1.3.1.build.log:######## retval=success
_5034-small__/termcap-1.3.1.build.log:######## retval=success
_5034-small__/termcap-1.3.1.build.log:######## retval=success
_5034-small__/termcap-1.3.1.build.log:######## retval=success

:-1:

genio commented 1 year ago

I got a response from the readline group and they have a patch for that already:

The latest push to the readline devel git branch has a fix for this and a
couple other cosmetic __MINGW32__ errors. It's basically that MINGW lacks
fd_set, and you need to be diligent about checking for it.

http://git.savannah.gnu.org/cgit/readline.git/commit/?h=devel&id=c95d589c5ad4710697a12dc2a2dbf998678de1e1

Chet
shawnlaffan commented 1 year ago

Other updates:

5 Proj is probably not needed any more

6 cfitsio builds but lacks the dllsuffix. It might also not be needed any more.

7 fontconfig compiles with a few updates

8 libiconv builds with the latest version (1.17)

I can submit PRs for each of these, or one omnibus PR with one change per distribution. Just let me know.

I think that leaves libhdf, libgdbm, libsodium and libssh with fails.

libsodium has an existing alien and might not be needed? It is not in the 5.32 distribution but perhaps there are plans for it.

(Edited 2022-10-17: s/libgdm/libgdbm/)

genio commented 1 year ago

A PR per item would be nice! Thank you for all of your help

shawnlaffan commented 1 year ago

Not a problem.

PRs in #9 (libiconv) and #10 (fontconfig).

I've held off on cfitsio and proj. The latest cfitsio does not append the DLLSUFFIX and appears not to be needed anyway. I don't think proj is needed in the Strawberry c dir and the build for version 9 requires an additional dependency (sqlite).

shawnlaffan commented 1 year ago

HDF summary in #11, PR in #12

shawnlaffan commented 1 year ago

libssl and libssh2 summary is in #14

shawnlaffan commented 1 year ago

freeglut can be built, summary is in #15 netcdf 4.9.0 can be built, summary is in #16

PRs are yet to be worked up.

shawnlaffan commented 1 year ago

Another for my PR list: libhdf5 can be updated to 1.13.2 https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.13/hdf5-1.13.2/src/hdf5-1.13.2.tar.gz

The build has many fewer warnings.

genio commented 1 year ago

Much closer:

_5034__/cfitsio-3.47.build.log:######## retval=FAILURE
_5034__/cfitsio-3.47.build.log:######## retval=success
_5034__/db-6.2.38.build.log:######## retval=success
_5034__/db-6.2.38.build.log:######## retval=success
_5034__/db-6.2.38.build.log:######## retval=success
_5034__/expat-2.2.6.build.log:######## retval=success
_5034__/expat-2.2.6.build.log:######## retval=success
_5034__/expat-2.2.6.build.log:######## retval=success
_5034__/expat-2.2.6.build.log:######## retval=success
_5034__/fftw-3.3.8.build.log:######## retval=success
_5034__/fftw-3.3.8.build.log:######## retval=success
_5034__/fftw-3.3.8.build.log:######## retval=success
_5034__/fftw-3.3.8.build.log:######## retval=success
_5034__/fftw-3.3.8.build.log:######## retval=success
_5034__/fftw-3.3.8.build.log:######## retval=success
_5034__/fftw-3.3.8.build.log:######## retval=success
_5034__/fftw-3.3.8.build.log:######## retval=success
_5034__/fftw-3.3.8.build.log:######## retval=success
_5034__/fontconfig-2.13.1.build.log:######## retval=success
_5034__/fontconfig-2.13.1.build.log:######## retval=success
_5034__/fontconfig-2.13.1.build.log:######## retval=success
_5034__/fontconfig-2.13.1.build.log:######## retval=success
_5034__/freeglut-3.4.0.build.log:######## retval=success
_5034__/freeglut-3.4.0.build.log:######## retval=success
_5034__/freeglut-3.4.0.build.log:######## retval=success
_5034__/freetype-2.10.0.build.log:######## retval=success
_5034__/freetype-2.10.0.build.log:######## retval=success
_5034__/freetype-2.10.0.build.log:######## retval=success
_5034__/fribidi-1.0.12.build.log:######## retval=success
_5034__/fribidi-1.0.12.build.log:######## retval=success
_5034__/fribidi-1.0.12.build.log:######## retval=success
_5034__/gdbm-1.18.build.log:######## retval=success
_5034__/gdbm-1.18.build.log:######## retval=success
_5034__/gdbm-1.18.build.log:######## retval=FAILURE
_5034__/gdbm-1.18.build.log:######## retval=FAILURE
_5034__/gdbm-1.18.build.log:######## retval=FAILURE
_5034__/giflib-5.1.9.build.log:######## retval=success
_5034__/giflib-5.1.9.build.log:######## retval=success
_5034__/giflib-5.1.9.build.log:######## retval=success
_5034__/gmp-6.2.1.build.log:######## retval=success
_5034__/gmp-6.2.1.build.log:######## retval=success
_5034__/gmp-6.2.1.build.log:######## retval=success
_5034__/gmp-6.2.1.build.log:######## retval=success
_5034__/gnuplot-5.2.6.build.log:######## retval=success
_5034__/gnuplot-5.2.6.build.log:######## retval=success
_5034__/gnuplot-5.2.6.build.log:######## retval=success
_5034__/graphite2-1.3.13.build.log:######## retval=success
_5034__/graphite2-1.3.13.build.log:######## retval=success
_5034__/graphite2-1.3.13.build.log:######## retval=success
_5034__/gsl-2.5.build.log:######## retval=success
_5034__/gsl-2.5.build.log:######## retval=success
_5034__/gsl-2.5.build.log:######## retval=success
_5034__/gsl-2.5.build.log:######## retval=success
_5034__/harfbuzz-2.3.1.build.log:######## retval=success
_5034__/harfbuzz-2.3.1.build.log:######## retval=success
_5034__/harfbuzz-2.3.1.build.log:######## retval=success
_5034__/hdf-4.2.15.build.log:######## retval=success
_5034__/hdf-4.2.15.build.log:######## retval=success
_5034__/hdf-4.2.15.build.log:######## retval=success
_5034__/hdf5-1.12.2.build.log:######## retval=success
_5034__/hdf5-1.12.2.build.log:######## retval=success
_5034__/hdf5-1.12.2.build.log:######## retval=success
_5034__/jpeg-9c.build.log:######## retval=success
_5034__/jpeg-9c.build.log:######## retval=success
_5034__/jpeg-9c.build.log:######## retval=success
_5034__/jpeg-9c.build.log:######## retval=success
_5034__/lapack-3.8.0.build.log:######## retval=success
_5034__/lapack-3.8.0.build.log:######## retval=success
_5034__/lapack-3.8.0.build.log:######## retval=success
_5034__/libcerf-1.3.build.log:######## retval=success
_5034__/libcerf-1.3.build.log:######## retval=success
_5034__/libcerf-1.3.build.log:######## retval=success
_5034__/libffi-3.2.1.build.log:######## retval=success
_5034__/libffi-3.2.1.build.log:######## retval=success
_5034__/libffi-3.2.1.build.log:######## retval=success
_5034__/libgd-2.2.5.build.log:######## retval=success
_5034__/libgd-2.2.5.build.log:######## retval=success
_5034__/libgd-2.2.5.build.log:######## retval=success
_5034__/libiconv-1.17.build.log:######## retval=success
_5034__/libiconv-1.17.build.log:######## retval=success
_5034__/libiconv-1.17.build.log:######## retval=success
_5034__/libiconv-1.17.build.log:######## retval=success
_5034__/libidn2-2.1.1.build.log:######## retval=success
_5034__/libidn2-2.1.1.build.log:######## retval=success
_5034__/libpng-1.6.37.build.log:######## retval=success
_5034__/libpng-1.6.37.build.log:######## retval=success
_5034__/libpng-1.6.37.build.log:######## retval=success
_5034__/libpng-1.6.37.build.log:######## retval=success
_5034__/libsodium-1.0.17.build.log:######## retval=success
_5034__/libsodium-1.0.17.build.log:######## retval=FAILURE
_5034__/libsodium-1.0.17.build.log:######## retval=FAILURE
_5034__/libssh2-1.10.0.build.log:######## retval=success
_5034__/libssh2-1.10.0.build.log:######## retval=success
_5034__/libssh2-1.10.0.build.log:######## retval=FAILURE
_5034__/libssh2-1.10.0.build.log:######## retval=success
_5034__/libunistring-0.9.10.build.log:######## retval=success
_5034__/libunistring-0.9.10.build.log:######## retval=success
_5034__/libunistring-0.9.10.build.log:######## retval=success
_5034__/libuv-v1.28.0.build.log:######## retval=success
_5034__/libuv-v1.28.0.build.log:######## retval=success
_5034__/libuv-v1.28.0.build.log:######## retval=success
_5034__/libxml2-2.9.9.build.log:######## retval=success
_5034__/libxml2-2.9.9.build.log:######## retval=success
_5034__/libxml2-2.9.9.build.log:######## retval=success
_5034__/libXpm-3.5.12.build.log:######## retval=success
_5034__/libxslt-1.1.33.build.log:######## retval=success
_5034__/libxslt-1.1.33.build.log:######## retval=success
_5034__/libxslt-1.1.33.build.log:######## retval=success
_5034__/libxslt-1.1.33.build.log:######## retval=success
_5034__/libxslt-1.1.33.build.log:######## retval=success
_5034__/mpc-1.1.0.build.log:######## retval=success
_5034__/mpc-1.1.0.build.log:######## retval=success
_5034__/mpc-1.1.0.build.log:######## retval=success
_5034__/mpc-1.1.0.build.log:######## retval=success
_5034__/mpfr-4.0.2.build.log:######## retval=success
_5034__/mpfr-4.0.2.build.log:######## retval=success
_5034__/mpfr-4.0.2.build.log:######## retval=success
_5034__/mpfr-4.0.2.build.log:######## retval=success
_5034__/netcdf-c-4.9.0.build.log:######## retval=success
_5034__/netcdf-c-4.9.0.build.log:######## retval=success
_5034__/netcdf-c-4.9.0.build.log:######## retval=FAILURE
_5034__/netcdf-c-4.9.0.build.log:######## retval=success
_5034__/openssl-1.1.1q.build.log:######## retval=success
_5034__/openssl-1.1.1q.build.log:######## retval=success
_5034__/openssl-1.1.1q.build.log:######## retval=success
_5034__/plplot-5.14.0.build.log:######## retval=success
_5034__/plplot-5.14.0.build.log:######## retval=success
_5034__/plplot-5.14.0.build.log:######## retval=success
_5034__/postgresql-15.1.build.log:######## retval=success
_5034__/postgresql-15.1.build.log:######## retval=success
_5034__/postgresql-15.1.build.log:######## retval=success
_5034__/postgresql-15.1.build.log:######## retval=success
_5034__/proj-9.1.0.build.log:######## retval=FAILURE
_5034__/proj-9.1.0.build.log:######## retval=FAILURE
_5034__/proj-9.1.0.build.log:######## retval=success
_5034__/readline-8.2.build.log:######## retval=success
_5034__/readline-8.2.build.log:######## retval=success
_5034__/readline-8.2.build.log:######## retval=success
_5034__/szip-2.1.1.build.log:######## retval=success
_5034__/szip-2.1.1.build.log:######## retval=success
_5034__/szip-2.1.1.build.log:######## retval=success
_5034__/t1lib-5.1.2.build.log:######## retval=success
_5034__/t1lib-5.1.2.build.log:######## retval=success
_5034__/t1lib-5.1.2.build.log:######## retval=success
_5034__/ta-lib-0.4.0.build.log:######## retval=success
_5034__/ta-lib-0.4.0.build.log:######## retval=success
_5034__/ta-lib-0.4.0.build.log:######## retval=success
_5034__/termcap-1.3.1.build.log:######## retval=success
_5034__/termcap-1.3.1.build.log:######## retval=success
_5034__/termcap-1.3.1.build.log:######## retval=success
_5034__/termcap-1.3.1.build.log:######## retval=success
_5034__/termcap-1.3.1.build.log:######## retval=success
_5034__/termcap-1.3.1.build.log:######## retval=success
_5034__/termcap-1.3.1.build.log:######## retval=success
_5034__/tiff-4.0.10.build.log:######## retval=success
_5034__/tiff-4.0.10.build.log:######## retval=success
_5034__/tiff-4.0.10.build.log:######## retval=success
_5034__/tiff-4.0.10.build.log:######## retval=success
_5034__/xz-5.2.4.build.log:######## retval=success
_5034__/xz-5.2.4.build.log:######## retval=success
_5034__/xz-5.2.4.build.log:######## retval=success
_5034__/xz-5.2.4.build.log:######## retval=success
_5034__/zlib-1.2.11.build.log:######## retval=success
shawnlaffan commented 1 year ago

This is good to see.

Proj needs sqlite, which is not currently in the build system. My notes are in #5 libgdbm 1.19 builds but with some failures for NDBM and ODBM. These might not matter (see #13 ). cfitsio is probably not needed in the c\bin dir - see comments in #6

I have not tried building libsodium.

shawnlaffan commented 1 year ago

libsodium builds successfully with CPPFLAGS="-D_FORTIFY_SOURCE=0".

Linking to libssp or adding -fstack-protector might be a better approach: https://github.com/msys2/MINGW-packages/issues/5868#issuecomment-543871523

There is also this news item which suggests the problem will disappear with a more recent mingw: https://www.msys2.org/news/#2022-10-10-libssp-is-no-longer-required

That still leaves the addition of the DLLSUFFIX to the dll name, though.

shawnlaffan commented 1 year ago

libsodium builds with gcc-13, and has the DLLSUFFIX.

shawnlaffan commented 1 year ago

Srawberry perl 5.36.1 and 5.38.0 have been released. I think this issue can be closed now.

shawnlaffan commented 9 months ago

And finally closing