[C++] C++ cookbooks are failing to find zlib

[raulcd]

C++ cookbooks fail with (https://github.com/apache/arrow-cookbook/actions/runs/5977593937/job/16217896402):

-- Could NOT find ZLIB (missing: ZLIB_LIBRARY) (found version "1.2.11")
-- Could NOT find ZLIB (missing: ZLIB_LIBRARY) (found version "1.2.11")
...
...
... OTHER NOT RELEVANT LOGS
...
...
CMake Error at /usr/share/miniconda/envs/cookbook-cpp-dev/lib/cmake/grpc/gRPCTargets.cmake:61 (set_target_properties):
  The link interface of target "gRPC::address_sorting" contains:
-- Generating done (0.0s)

    ZLIB::ZLIB

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

Call Stack (most recent call first):
  /usr/share/miniconda/envs/cookbook-cpp-dev/lib/cmake/grpc/gRPCConfig.cmake:23 (include)
  CMakeLists.txt:81 (find_package)

CMake Error at /usr/share/miniconda/envs/cookbook-cpp-dev/lib/cmake/grpc/gRPCTargets.cmake:70 (set_target_properties):
  The link interface of target "gRPC::gpr" contains:

    ZLIB::ZLIB

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

Call Stack (most recent call first):
  /usr/share/miniconda/envs/cookbook-cpp-dev/lib/cmake/grpc/gRPCConfig.cmake:23 (include)
  CMakeLists.txt:81 (find_package)

CMake Error at /usr/share/miniconda/envs/cookbook-cpp-dev/lib/cmake/grpc/gRPCTargets.cmake:97 (set_target_properties):
  The link interface of target "gRPC::grpc++" contains:

    ZLIB::ZLIB

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

Call Stack (most recent call first):
  /usr/share/miniconda/envs/cookbook-cpp-dev/lib/cmake/grpc/gRPCConfig.cmake:23 (include)
  CMakeLists.txt:81 (find_package)

CMake Error at /usr/share/miniconda/envs/cookbook-cpp-dev/lib/cmake/grpc/gRPCTargets.cmake:169 (set_target_properties):
  The link interface of target "gRPC::upb" contains:

    ZLIB::ZLIB

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

Call Stack (most recent call first):
  /usr/share/miniconda/envs/cookbook-cpp-dev/lib/cmake/grpc/gRPCConfig.cmake:23 (include)
  CMakeLists.txt:81 (find_package)

CMake Generate step failed.  Build files cannot be regenerated correctly.

[amoeba]

I wanted to fix this but I'm not sure I'm familiar enough with how conda works yet. Here's what I found in case it helps:

I first confirmed I get the same failure as above and then fixed it by installing zlib manually from conda after I create my env. So I guess that's a short-term fix. Since zlib isn't a new requirement, I wondered why it wasn't being installed when CI creates a new conda env from cpp/dev.yml. When I do a dry run against it (conda env create -d -n foo -f cpp/dev.yml), zlib isn't listed whereas it is listed if I do dry run on macOS. Is there a good way to find out how a particular dep (zlib) gets pulled in?

[lidavidm]

It looks like it installs libzlib, which has libz.so, but not the header. zlib contains the actual header.

I suppose:

• Arrow depends on zlib at build time, but only libzlib at runtime.
• This is sufficient for anyone just linking to Arrow.
• But if you also want to link to zlib yourself, you need to also install the headers.
• I'm not sure what the policy of conda-forge is on transitive compilation dependencies: gRPC appears to require zlib as well if you want to build against gRPC, but of course if you're just using gRPC that's not necessary.
• So we should just explicitly list zlib. (I think.)

> jq .files $CONDA_PREFIX/conda-meta/libzlib-1.2.13-hd590300_5.json                                                                                                   (arrow-cookbook) 
[
  "lib/libz.so.1",
  "lib/libz.so.1.2.13"
]
> jq .files $CONDA_PREFIX/conda-meta/zlib-1.2.13-hd590300_5.json                                                                                                      (arrow-cookbook) 
[
  "include/zconf.h",
  "include/zlib.h",
  "lib/libz.a",
  "lib/libz.so",
  "lib/pkgconfig/zlib.pc"
]

[amoeba]

Thanks @lidavidm. I think explicitly listing zlib makes sense as a fix here. I'll send in a PR in a sec.

Since I assume this CI job worked at some point, I wanted to check to see what changed. If I create stable and dev conda environments for the cookbook locally, the stable env appears to get zlib via libgrpc so something must have changed between libgrpc 1.54.3 and 1.56.2.

cookbook-cpp (stable)

$ jq .depends $CONDA_PREFIX/conda-meta/libgrpc-1.54.3-hb20ce57_0.json
[
  "c-ares >=1.19.1,<2.0a0",
  "libabseil * cxx17*",
  "libabseil >=20230125.3,<20230126.0a0",
  "libgcc-ng >=12",
  "libprotobuf >=3.21.12,<3.22.0a0",
  "libstdcxx-ng >=12",
  "libzlib >=1.2.13,<1.3.0a0",
  "openssl >=3.1.1,<4.0a0",
  "re2 >=2023.3.2,<2023.3.3.0a0",
  "zlib"
]

cookbook-cpp-dev (dev)

$ jq .depends $CONDA_PREFIX/conda-meta/libgrpc-1.56.2-h3905398_1.json
[
  "c-ares >=1.19.1,<2.0a0",
  "libabseil * cxx17*",
  "libabseil >=20230125.3,<20230126.0a0",
  "libgcc-ng >=12",
  "libprotobuf >=4.23.3,<4.23.4.0a0",
  "libstdcxx-ng >=12",
  "libzlib >=1.2.13,<1.3.0a0",
  "openssl >=3.1.2,<4.0a0",
  "re2 >=2023.3.2,<2023.3.3.0a0"
]

[amoeba]

@raulcd do you want to look at https://github.com/apache/arrow-cookbook/pull/328?