zlib 1.2.12 is no longer available

[alex-virodov]

Hello. I am trying to run the docker build today, and I've hit the issue that https://zlib.net/zlib-1.2.12.tar.gz is no longer available. It seems that 1.2.12 was removed due to a security issue (see main page https://zlib.net/).

I am currently bumping zlib to 1.2.13, the build seems to make progress now.

Also, I had to hack getver.py, changing the first line to #!/usr/bin/python otherwise it wasn't working. Not sure if known issue.

[manthey]

You probably want to run python3 -u check_versions.py > versions.txt before doing the docker build to ensure all versions are up to date and exist. getver.py works within the docker build for me. Is it not working in the docker build for you?

[manthey]

Locally, we have a nightly build that runs https://github.com/girder/large_image_wheels/blob/master/rebuild.sh and then https://github.com/girder/large_image_wheels/blob/master/test_via_docker.py, and that currently passes.

[alex-virodov]

1. Is it possible it uses some cached version of zlib? The file the build process tries to download is missing (at least for me, easy to check if you follow the link to https://zlib.net/zlib-1.2.12.tar.gz - that's what the script tries to download for me and fails). I guess if you don't see an issue, that's fine, maybe it is something on my side.

2. as to getver.py, I am on windows, maybe that causes an issue somehow (docker should isolate of course). In any case, not a big deal for me, I am just experimenting.

3. To expand the scope a bit, maybe you can direct me to the right place. The reason I am building this container is because I am adding a support of new format to OpenSlide, and I would like to use that OpenSlide in DSA. However, when I just swap the .so with my own libopenslide.so (tried to build from my branch, head, or v3.4.1 tag + your patches), I get black tiles on .svs files (I can send screenshots). I am wondering if this is related to how you build openslide/libtiff/jpeg-turbo. I was thinking of "importing" my branch into this docker file and then building it with your dependency setup/patches. Does that sound right, or should I do something else/ask someone else?

[alex-virodov]

FYI:

(venv) ~/large_image_wheels$ python3 -u check_versions.py
...
zlib 1.2.13

So it seems check_versions.py finds the updated version. Maybe check_versions.py should be part of build instructions? Your call of course, please feel free to close the issue, it is resolved for me. Advice on general approach to building OpenSlide would still be welcomed, if available.

Thank you for the quick replies. Alex.

[manthey]

Yes, importing your branch and building sounds like a reasonable approach. Just swapping in your .so is going to try to load system libraries, not bundled libraries. As part of the docker build, auditwheel isolates libraries to add a hash that ensures you use the built libraries and not the system libraries. Without that, you need to have all appropriate system libraries (which means libtiff probably needs all of its dependencies, etc.

I haven't tried running docker build on anything other than Ubuntu systems. I'll spin up a Windows machine with WSL2 and see what happens.

[alex-virodov]

Thank you, got it. My built version actually works (I overwrite the hash-stamped file), so it finds all the dependencies, but probably not the ones built by this process.

I am using docker desktop with WSL2, I just executed docker build --force-rm -t girder/large_image_wheels . on today's repository clone. What I observed is that it tries to get 'pkg-config-', and when I ran the container build to that point and went in manually, I tried to run getver.py pkg-config and got:

PS C:\Users\avirodov> docker run -it --rm girder/large_image_wheels bash
[root@515c9b7a54cb build]# ls
glymur.setup.py  mapnik_proj_transform.cpp.patch  openslide-init.patch            versions.txt
log.txt          mapnik_setup.py.patch            openslide-vendor-mirax.c.patch
[root@515c9b7a54cb build]# getver.py pkg-config
: No such file or directory
[root@515c9b7a54cb build]# which getver.py
/usr/local/bin/getver.py
[root@515c9b7a54cb build]# vi /usr/local/bin/getver.py  <-- here I modified the shebang line to be #!/usr/bin/python
[root@515c9b7a54cb build]# python /usr/local/bin/getver.py
Traceback (most recent call last):
  File "/usr/local/bin/getver.py", line 15, in <module>
    ver = versions[sys.argv[1]]
IndexError: list index out of range
[root@515c9b7a54cb build]# python /usr/local/bin/getver.py pkg-config
0.29.2[root@515c9b7a54cb build]#

[alex-virodov]

Ok, it is windows issue. I converted endlines in getver.py to Unix format, and it works properly now.

Thank you for your time and replies. I will close the issue now, all my concerns were addressed.