Closed kloczek closed 2 years ago
jkeenan
commented
2 years ago Description Looks like perl at the moment cannot be built against zlib-ng (it is only known to me project which cannot ATM be build against zlib-ng)
Can you provide URLs for what you believe to be definitive sources and documentation for zlib-ng?
(I ask so that we can all be on the same page with what is essentially a new feature request.)
Thank you very much. Jim Keenan
kloczek
commented
2 years ago https://github.com/zlib-ng/zlib-ng/ zlib-ng is available as installeable package in most of the major Linux distributions.
hvds
commented
2 years ago @kloczek I think you may need to give more details of how you are building zlib-ng and how you are building perl, since a) I wasn't able to reproduce this, b) the errors you show appear to be internal errors within zlib.h (gzFile and ZNULL are both declared there).
I cloned the zlib-ng repository, then did:
git checkout 2.0.6 # appears to be latest version
./configure --prefix=/opt/zlib-ng-2.0.6 --zlib-compat
make ; make test ; make installI then configured and built perl with:
git checkout blead # latest code
./Configure -des -Dcc=gcc -Dprefix=/opt/scratch \
-A prepend:locincpth="/opt/zlib-ng-2.0.6/include " \
-A prepend:loclibpth="/opt/zlib-ng-2.0.6/lib "
make all.. which built ok (but by default builds Compress::Raw::Zlib using its own bundled zlib source).
I then did:
cd cpan/Compress-Raw-Zlib
make distclean
vi config.in # see belowthen rebuilt perl. This time it built using the supplied zlib-ng library, and again gave no compilation errors.
The config.in changes I made were:
BUILD_ZLIB = False
INCLUDE = /opt/zlib-ng-2.0.6/include
LIB = /opt/zlib-ng-2.0.6/lib.. and I verified the result so:
% ldd ../../lib/auto/Compress/Raw/Zlib/Zlib.so | grep zlib
libz.so.1 => /opt/zlib-ng-2.0.6/lib/libz.so.1 (0x00007f0dea7a6000)
% I'm building zlib-ng using cmake
%cmake \
-D WITH_GZFILEOP=ON \
-D WITH_MAINTAINER_WARNINGS=OFF \
-D WITH_NATIVE_INSTRUCTIONS=ON \
-D WITH_SANITIZERS=OFF \
-D ZLIB_COMPAT=ON \
-D ZLIB_ENABLE_TESTS=ON \
%{nil}I'm building zlib-ng using cmake
And which version of the zlib-ng code are you building? And how are you configuring perl? Oh, and what platform are you on?
Given my first attempt to reproduce it did not show any problem, it would make sense to give as much information as you can that might affect reproducibility.
kloczek
commented
2 years ago And which version of the zlib-ng code are you building?
I'm using latest 2.0.5.
If you will look closer on zlib-ng you can easyly grep for gzFile and you will be able to find that there is no prootypes or defines with that name.
If you are able to build with zlib-ng devel resources installed probably in your case you have zlib headers in /usr/include.
Just please make sure that really you don't have zlib.h in standard include path.
hvds
commented
2 years ago And which version of the zlib-ng code are you building?
I'm using latest 2.0.5. If you will look closer on zlib-ng you can easyly grep for
gzFileand you will be able to find that there is no prootypes or defines with that name.
gzFile is typedef'd in zlib.h and in zlib-ng.h: https://github.com/zlib-ng/zlib-ng/blob/c69f78bc5e18a0f6de2dcbd8af863f59a14194f0/zlib.h#L1279 and https://github.com/zlib-ng/zlib-ng/blob/c69f78bc5e18a0f6de2dcbd8af863f59a14194f0/zlib-ng.h#L1294 (that's the commit tagged as 2.0.5).
In zlib.h the underlying gzFile_s structure is declared whenever Z_SOLO is not defined. That's also how it works in the zlib.h bundled with perl under cpan/Compress-Raw-Zlib/zlib-src/, and also in my system zlib.h under /usr/include.
I think you're going to have to do some diagnosis yourself on this one to work out what is going wrong for your build.
kloczek
commented
2 years ago In zlib.h the underlying
gzFile_sstructure is declared wheneverZ_SOLOis not defined. That's also how it works in the zlib.h bundled with perl undercpan/Compress-Raw-Zlib/zlib-src/, and also in my system zlib.h under/usr/include.
Please look one more time on quoted output.
What is missing is not gzFile_s but gzFile type definition
[tkloczko@devel-g2v SPECS]$ grep gzFile -w /usr/include/zlib.h
typedef struct gzFile_s *gzFile; /* semi-opaque gzip file descriptor */
Z_EXTERN gzFile Z_EXPORT gzopen(const char *path, const char *mode);
insufficient memory to allocate the gzFile state, or if an invalid mode was
Z_EXTERN gzFile Z_EXPORT gzdopen(int fd, const char *mode);
Associate a gzFile with the file descriptor fd. File descriptors are
The next call of gzclose on the returned gzFile will also close the file
gzFile state, if an invalid mode was specified (an 'r', 'w', or 'a' was not
Z_EXTERN int Z_EXPORT gzbuffer(gzFile file, unsigned size);
Z_EXTERN int Z_EXPORT gzsetparams(gzFile file, int level, int strategy);
Z_EXTERN int Z_EXPORT gzread(gzFile file, void *buf, unsigned len);
Z_EXTERN size_t Z_EXPORT gzfread (void *buf, size_t size, size_t nitems, gzFile file);
Z_EXTERN int Z_EXPORT gzwrite(gzFile file, void const *buf, unsigned len);
Z_EXTERN size_t Z_EXPORT gzfwrite(void const *buf, size_t size, size_t nitems, gzFile file);
Z_EXTERN int Z_EXPORTVA gzprintf(gzFile file, const char *format, ...);
Z_EXTERN int Z_EXPORT gzputs(gzFile file, const char *s);
Z_EXTERN char * Z_EXPORT gzgets(gzFile file, char *buf, int len);
Z_EXTERN int Z_EXPORT gzputc(gzFile file, int c);
Z_EXTERN int Z_EXPORT gzgetc(gzFile file);
Z_EXTERN int Z_EXPORT gzungetc(int c, gzFile file);
Z_EXTERN int Z_EXPORT gzflush(gzFile file, int flush);
Z_EXTERN z_off_t Z_EXPORT gzseek (gzFile file, z_off_t offset, int whence);
Z_EXTERN int Z_EXPORT gzrewind(gzFile file);
Z_EXTERN z_off_t Z_EXPORT gztell(gzFile file);
Z_EXTERN z_off_t Z_EXPORT gzoffset(gzFile file);
Z_EXTERN int Z_EXPORT gzeof(gzFile file);
Z_EXTERN int Z_EXPORT gzdirect(gzFile file);
Z_EXTERN int Z_EXPORT gzclose(gzFile file);
Z_EXTERN int Z_EXPORT gzclose_r(gzFile file);
Z_EXTERN int Z_EXPORT gzclose_w(gzFile file);
Z_EXTERN const char * Z_EXPORT gzerror(gzFile file, int *errnum);
Z_EXTERN void Z_EXPORT gzclearerr(gzFile file);
Z_EXTERN int Z_EXPORT gzgetc_(gzFile file); /* backward compatibility */
Z_EXTERN gzFile Z_EXPORT gzopen64(const char *, const char *);
Z_EXTERN z_off64_t Z_EXPORT gzseek64(gzFile, z_off64_t, int);
Z_EXTERN z_off64_t Z_EXPORT gztell64(gzFile);
Z_EXTERN z_off64_t Z_EXPORT gzoffset64(gzFile);
Z_EXTERN gzFile Z_EXPORT gzopen64(const char *, const char *);
Z_EXTERN z_off_t Z_EXPORT gzseek64(gzFile, z_off_t, int);
Z_EXTERN z_off_t Z_EXPORT gztell64(gzFile);
Z_EXTERN z_off_t Z_EXPORT gzoffset64(gzFile);
Z_EXTERN gzFile Z_EXPORT gzopen(const char *, const char *);
Z_EXTERN z_off_t Z_EXPORT gzseek(gzFile, z_off_t, int);
Z_EXTERN z_off_t Z_EXPORT gztell(gzFile);
Z_EXTERN z_off_t Z_EXPORT gzoffset(gzFile);
Z_EXTERN gzFile Z_EXPORT gzopen_w(const wchar_t *path, const char *mode);
Z_EXTERN int Z_EXPORTVA gzvprintf(gzFile file, const char *format, va_list va);
[tkloczko@devel-g2v SPECS]$ grep gzFile -w /usr/include/zlib.h | grep type
typedef struct gzFile_s *gzFile; /* semi-opaque gzip file descriptor */Also zlib-ng.h is not installed. That is all what I have about zlib-ng devel resources
[tkloczko@devel-g2v SPECS]$ rpm -ql zlib-ng-devel
/usr/include/zconf.h
/usr/include/zlib.h
/usr/lib64/libz.so
/usr/lib64/pkgconfig/zlib.pcPlease look one more time on quoted output. What is missing is not
gzFile_sbutgzFiletype definition[tkloczko@devel-g2v SPECS]$ grep gzFile -w /usr/include/zlib.h typedef struct gzFile_s *gzFile; /* semi-opaque gzip file descriptor */
This is the type definition. You will need to discover why when you compile this it is not picking up this type definition, I can't do that for you.
kloczek
commented
2 years ago This is the type definition. You will need to discover why when you compile this it is not picking up this type definition, I can't do that for you.
How can I trace that? 🤔
kloczek
commented
2 years ago FYI: I've opened https://github.com/zlib-ng/zlib-ng/issues/1234
mtl1979
commented
2 years ago Compiler flags include -DZ_SOLO which disables support for gzip-related functions and macros. Removing that should allow compilation to continue... With current development version of zlib-ng, those functions are correctly guarded with check for Z_SOLO not being defined.
nmoinvaz
commented
2 years ago Also zlib-ng.h is not installed.
zlib-ng.h will not be installed if you are using -DZLIB_COMPAT=ON. You'll only get zlib.h.
kloczek
commented
2 years ago zlib-ng.h will not be installed if you are using -DZLIB_COMPAT=ON. You'll only get zlib.h.
I'm puzzled .. so using -DZLIB_COMPAT=ON does not mean build and install zlib-ng in zlib compatibility mode? 🤔
mtl1979
commented
2 years ago zlib-ng.h will not be installed if you are using -DZLIB_COMPAT=ON. You'll only get zlib.h.
I'm puzzled .. so using
-DZLIB_COMPAT=ONdoes not mean build and install zlib-ng in zlib compatibility mode? 🤔
Incorrect. It just changes the header filenames to match what zlib has.
kloczek
commented
2 years ago Just started upgradig my perl.spec for 5.36.0 and found that now there is no build issues however loks like test suite is failing. zlib-ng 2.0.6
kloczek
commented
2 years ago @nmoinvaz can you have look on that issue?
nmoinvaz
commented
2 years ago I don't know perl well enough.
kloczek
commented
2 years ago I'm almost sure that in tis case you don't need to know to much about perl😋
Compress::Raw::Zlib module used DSO which on one end provides few symbols for that perl module, and on the other end is using libz symbols.
That DSO is written in C https://github.com/Perl/perl5/blob/blead/cpan/Compress-Raw-Zlib/Zlib.xs
.xs functionally is similar to python .pyx file .. both are transformed on build time to raw .c and compiled.
According to error messages test code provides output which is different to what is expected (I'm still not sure about which one part of the compressed content that unit complains. Compress::Raw::Zlib maintainer probably can provide more about propose of those two units (can we have some expertise from perl end about those units? 🤔 )
nmoinvaz
commented
2 years ago It is likely the perl tests need to be updated. Often tests from other projects rely on a bit for bit identical deflate stream (to zlib) being produced and we don't make that guarantee. See also https://github.com/zlib-ng/zlib-ng/discussions/905.
nmoinvaz
commented
2 years ago # Failed test ' CINFO matches'
# at t/005defhdr.t line 184.
# got: '7'
# expected: '5' CINFO (Compression info)
For CM = 8, CINFO is the base-2 logarithm of the LZ77 window
size, minus eight (CINFO=7 indicates a 32K window size). Values
of CINFO above 7 are not allowed in this version of the
specification. CINFO is not defined in this specification for
CM not equal to 8.CINFO should be checked that it is <= 7, unless you are explicitly specifying window size when creating the deflate stream. I don't see anywhere where it is setting the window size when creating the stream.
nmoinvaz
commented
2 years ago # Failed test ' FCHECK matches'
# at t/005defhdr.t line 187.
# got: '1'
# expected: '9'Because CINFO is different I suspect FCHECK is not what is expected either. It would be better to do FCHECK calculation manually to verify than to rely on fixed value.
pmqs
commented
2 years ago I have a change in progress to add support for zlib-ng running in either native or compat mode that is (almost) working on my dev box.
I'm using ticket https://github.com/pmqs/Compress-Raw-Zlib/issues/9 to track this work.
pmqs
commented
2 years ago This ticket can be closed. Support for zlib-ng added in Compress::Raw::Zlib 2.200. Uploaded to CPAN (https://metacpan.org/dist/Compress-Raw-Zlib)
jkeenan
commented
2 years ago I have a change in progress to add support for zlib-ng running in either native or compat mode that is (almost) working on my dev box.
I'm using ticket pmqs/Compress-Raw-Zlib#9 to track this work.
Since we have heard no objection to closing this ticket in the 3 months since @pmqs's comment, closing this now.
Description Loomks like perl at the moment cannot be build against zlib-ng (it is only knowjn to me project which cannot ATM be build against zlib-ng)
Steps to Reproduce
Expected behavior it would be possible to build perl against zlib-ng.
Compile time errors: