png/zlib related error while running cmake on "Installing USD_FROM_GLTF" step

[simonseo]

Hi all,

Thanks for the great program. I'm trying to install ufg on a fresh Ubuntu 14.04 machine on AWS EC2. (I was running into too many problems on Ubuntu 18.04 so I downgraded to the same OS that USD's Travis CI uses)

I'm getting the following error and I would appreciate any help!

ubuntu@ip-172-31-36-22:~/Downloads/usd_from_gltf$ python tools/ufginstall/ufginstall.py ~/Downloads/ufg_build /usr/local/USD --force png
Install Settings:
  Source Directory:   /home/ubuntu/Downloads/usd_from_gltf
  Install Directory:  /home/ubuntu/Downloads/ufg_build
  USD Directory:      /usr/local/USD
  Download Directory: /home/ubuntu/Downloads/ufg_build/src
  Build Directory:    /home/ubuntu/Downloads/ufg_build/build
  Build Config:       Release
  CMake Generator:    Default
  Already Installed:  DRACO, GIF, JPG, JSON, STB_IMAGE, TCLAP
  Installing:         PNG, USD_FROM_GLTF

-------- Installing PNG --------
  PNG: Downloading https://download.sourceforge.net/libpng/libpng-1.6.37.tar.gz to /home/ubuntu/Downloads/ufg_build/src/png.zip
  PNG: Unpacking to: /home/ubuntu/Downloads/ufg_build/src/libpng-1.6.37
  PNG: CWD: /home/ubuntu/Downloads/ufg_build/build/libpng-1.6.37
  PNG: Run: cmake /home/ubuntu/Downloads/ufg_build/src/libpng-1.6.37 -DCMAKE_INSTALL_PREFIX=/home/ubuntu/Downloads/ufg_build -DCMAKE_PREFIX_PATH=/home/ubuntu/Downloads/ufg_build -DCMAKE_POSITION_INDEPENDENT_CODE=1
  PNG: CWD: /home/ubuntu/Downloads/ufg_build/build/libpng-1.6.37
  PNG: Run: cmake --build . --config Release --target install --

-------- Installing USD_FROM_GLTF --------
  USD_FROM_GLTF: CWD: /home/ubuntu/Downloads/ufg_build/build/usd_from_gltf
  USD_FROM_GLTF: Run: cmake /home/ubuntu/Downloads/usd_from_gltf -DCMAKE_INSTALL_PREFIX=/home/ubuntu/Downloads/ufg_build -DCMAKE_PREFIX_PATH=/home/ubuntu/Downloads/ufg_build -DUSD_DIR=/usr/local/USD
  USD_FROM_GLTF: CWD: /home/ubuntu/Downloads/ufg_build/build/usd_from_gltf
  USD_FROM_GLTF: Run: cmake --build . --config Release --target install --
[ 30%] Built target gltf
[ 40%] Built target common
Scanning dependencies of target process
[ 42%] Building CXX object process/CMakeFiles/process.dir/image_png.cc.o
[ 45%] Linking CXX static library libprocess.a
[ 71%] Built target process
[ 88%] Built target convert
[ 90%] Linking CXX executable usd_from_gltf
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(png.c.o): In function `png_reset_crc':
png.c:(.text+0x1e2): undefined reference to `crc32'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(png.c.o): In function `png_calculate_crc':
png.c:(.text+0x29f): undefined reference to `crc32'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(png.c.o): In function `png_reset_zstream':
png.c:(.text+0x1779): undefined reference to `inflateReset'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(png.c.o): In function `png_compare_ICC_profile_with_sRGB':
png.c:(.text+0x3c6d): undefined reference to `adler32'
png.c:(.text+0x3c87): undefined reference to `adler32'
png.c:(.text+0x3cc6): undefined reference to `crc32'
png.c:(.text+0x3ce0): undefined reference to `crc32'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(pngread.c.o): In function `png_read_destroy':
pngread.c:(.text+0x1794): undefined reference to `inflateEnd'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(pngrutil.c.o): In function `png_inflate_claim':
pngrutil.c:(.text+0x8db): undefined reference to `inflateReset2'
pngrutil.c:(.text+0x901): undefined reference to `inflateInit2_'
pngrutil.c:(.text+0x94f): undefined reference to `inflateValidate'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(pngrutil.c.o): In function `png_zlib_inflate':
pngrutil.c:(.text+0xa0e): undefined reference to `inflate'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(pngrutil.c.o): In function `png_decompress_chunk':
pngrutil.c:(.text+0xe2a): undefined reference to `inflateReset'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(pngwrite.c.o): In function `png_write_destroy':
pngwrite.c:(.text+0x18d8): undefined reference to `deflateEnd'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(pngwutil.c.o): In function `png_deflate_claim':
pngwutil.c:(.text+0x997): undefined reference to `deflateEnd'
pngwutil.c:(.text+0xa24): undefined reference to `deflateReset'
pngwutil.c:(.text+0xa56): undefined reference to `deflateInit2_'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(pngwutil.c.o): In function `png_text_compress':
pngwutil.c:(.text+0xd0b): undefined reference to `deflate'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(pngwutil.c.o): In function `png_compress_IDAT':
pngwutil.c:(.text+0x15d8): undefined reference to `deflate'
collect2: error: ld returned 1 exit status
make[2]: *** [usd_from_gltf/usd_from_gltf] Error 1
make[1]: *** [usd_from_gltf/CMakeFiles/usd_from_gltf.dir/all] Error 2
make: *** [all] Error 2

  USD_FROM_GLTF: ERROR: Command failed: cmake --build . --config Release --target install --
  USD_FROM_GLTF: See log at: /home/ubuntu/Downloads/ufg_build/build/usd_from_gltf/log.txt

As I understand it, it seems that the png library cannot find zlib. So far I've tried installing zlib1g and zlib1g-dev via apt and installing from source (http://zlib.net/zlib-1.2.11.tar.gz). I added the most obvious directories to PATH:

/bin
/sbin
/usr/bin
/usr/include
/usr/local/bin
/usr/local/games
/usr/local/lib/
/usr/local/sbin
/usr/local/USD/bin
/usr/sbin

Here are all the zlib/libz related files I could find:

ubuntu@ip-172-31-36-22:~/Downloads/usd_from_gltf$ ls /usr/local/lib/
cmake/                         libboost_graph.so              libboost_locale.so.1           libboost_python34.so           libz.so
libboost_fiber.so              libboost_graph.so.1            libboost_locale.so.1.71        libboost_python34.so.1         libz.so.1
libboost_fiber.so.1            libboost_graph.so.1.71         libboost_locale.so.1.71.0      libboost_python34.so.1.71      libz.so.1.2.11
libboost_fiber.so.1.71         libboost_graph.so.1.71.0       libboost_log.so                libboost_python34.so.1.71.0    pkgconfig/
libboost_fiber.so.1.71.0       libboost_iostreams.so          libboost_log.so.1              libboost_regex.so              python2.7/
libboost_filesystem.so         libboost_iostreams.so.1        libboost_log.so.1.71           libboost_regex.so.1            python3.4/
libboost_filesystem.so.1       libboost_iostreams.so.1.71     libboost_log.so.1.71.0         libboost_regex.so.1.71         
libboost_filesystem.so.1.71    libboost_iostreams.so.1.71.0   libboost_python.so             libboost_regex.so.1.71.0       
libboost_filesystem.so.1.71.0  libboost_locale.so             libboost_python34.a            libz.a                         

ubuntu@ip-172-31-36-22:~/Downloads/usd_from_gltf$ find /usr | grep zlib.h
/usr/src/linux-headers-3.13.0-170/include/linux/zlib.h
/usr/src/linux-headers-3.13.0-170-generic/include/config/jffs2/zlib.h
/usr/src/linux-headers-3.13.0-170-generic/include/config/squashfs/zlib.h
/usr/src/linux-headers-3.13.0-170-generic/include/config/crypto/zlib.h
/usr/src/linux-headers-3.13.0-170-generic/include/config/ubifs/fs/zlib.h
/usr/src/linux-headers-3.13.0-170-generic/include/linux/zlib.h
/usr/include/zlib.h
/usr/local/USD/src/tbb-4.4.6/examples/graph/fgbzip2/bzlib.h
/usr/local/USD/src/boost_1_55_0/boost/iostreams/filter/zlib.hpp
/usr/local/USD/src/boost_1_55_0/boost/iostreams/detail/config/zlib.hpp
/usr/local/USD/include/boost/iostreams/filter/zlib.hpp
/usr/local/USD/include/boost/iostreams/detail/config/zlib.hpp
/usr/local/include/zlib.h
/usr/local/include/boost/beast/zlib.hpp
/usr/local/include/boost/beast/zlib/zlib.hpp
/usr/local/include/boost/iostreams/filter/zlib.hpp
/usr/local/include/boost/iostreams/detail/config/zlib.hpp
/usr/share/doc/zlib1g-dev/examples/zlib_how.html

ubuntu@ip-172-31-36-22:~/Downloads/usd_from_gltf$ /sbin/ldconfig -p| grep libz.so
    libz.so.1 (libc6,x86-64) => /usr/local/lib/libz.so.1
    libz.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libz.so.1
    libz.so (libc6,x86-64) => /usr/local/lib/libz.so

I'm not sure what LIBRARY_PATH and LD_LIBRARY_PATH variables do so I haven't really touched those.

Again, any pointers would be appreciated!

Thanks.

[C0DED00D]

That is strange. We are using CMake, which is supposed to handle finding and linking in libraries appropriately.

If you look in: usd_from_gltf/process/CMakeLists.txt

Before: if (MSVC) target_link_libraries(process

Can you add: message("${ZLIB_LIBRARIES}")

That will at least tell you where CMake thinks your Zlib libraries are. Perhaps for some reason the install was incomplete since it appears to be finding the headers and a directory where it thinks zlib lives. (Otherwise it exists on the first line of that CMake file)

Thanks, and sorry for the trouble!

[simonseo]

Thanks for the quick reply @C0DED00D,

so I made the following edit:

# usd_from_gltf/process/CMakeLists.txt 
# ...
if (NOT TURBOJPEG_LIBRARY)
  message(FATAL_ERROR "Libturbojpeg not found.")
endif (NOT TURBOJPEG_LIBRARY)

message("${ZLIB_LIBRARIES}")

if (MSVC)
target_link_libraries(process
  common
  "${ZLIB_LIBRARIES}"
  "${draco_LIBRARY_DIR}/dracodec.lib"
# ...

This is the last bit of the log:

# /home/ubuntu/Downloads/ufg_build/build/usd_from_gltf/log.txt

cmake /home/ubuntu/Downloads/usd_from_gltf -DCMAKE_INSTALL_PREFIX=/home/ubuntu/Downloads/ufg_build -DCMAKE_PREFIX_PATH=/home/ubuntu/Downloads/ufg_build -DUSD_DIR=/usr/local/USD
/usr/local/lib/libz.so
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/Downloads/ufg_build/build/usd_from_gltf

cmake --build . --config Release --target install --
[ 30%] Built target gltf
[ 40%] Built target common
Scanning dependencies of target process
[ 42%] Building CXX object process/CMakeFiles/process.dir/image_png.cc.o
[ 45%] Linking CXX static library libprocess.a
[ 71%] Built target process
[ 88%] Built target convert
[ 90%] Linking CXX executable usd_from_gltf
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(png.c.o): In function `png_reset_crc':
png.c:(.text+0x1e2): undefined reference to `crc32'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(png.c.o): In function `png_calculate_crc':
png.c:(.text+0x29f): undefined reference to `crc32'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(png.c.o): In function `png_reset_zstream':
png.c:(.text+0x1779): undefined reference to `inflateReset'
/home/ubuntu/Downloads/ufg_build/lib/libpng16.a(png.c.o): In function `png_compare_ICC_profile_with_sRGB':
png.c:(.text+0x3c6d): undefined reference to `adler32'
png.c:(.text+0x3c87): undefined reference to `adler32'
png.c:(.text+0x3cc6): undefined reference to `crc32'
png.c:(.text+0x3ce0): undefined reference to `crc32'
# ....

So from what I understood, /usr/local/lib/libz.so must be where CMake is looking for right? It seems that that file already exists.

ubuntu@ip-172-31-36-22:~/Downloads/usd_from_gltf$ ls -l /usr/local/lib/libz*
-rw-r--r-- 1 root root 166346 Oct 15 07:12 /usr/local/lib/libz.a
lrwxrwxrwx 1 root root     14 Oct 15 07:12 /usr/local/lib/libz.so -> libz.so.1.2.11
lrwxrwxrwx 1 root root     14 Oct 15 07:12 /usr/local/lib/libz.so.1 -> libz.so.1.2.11
-rwxr-xr-x 1 root root 129818 Oct 15 07:12 /usr/local/lib/libz.so.1.2.11

ubuntu@ip-172-31-36-22:~/Downloads/usd_from_gltf$ /usr/local/lib/libz.so
Segmentation fault (core dumped)

Which should CMake be looking for -- .so files or .h files?

[C0DED00D]

I believe it should be looking for the .so files. The zlib header seems to be defined, but at the linking stage, it can't find the symbol, which means the compiler is not loading the so for some reason.

Are you using a 64 bit version of Ubuntu? This is a bit of a stretch, but is it possible that the .so is 64 bit and usd_from_gltf is 32 bit or something like that?

[simonseo]

I'm on a 64 bit machine: Canonical, Ubuntu, 14.04 LTS, amd64 trusty. Everything was either downloaded via apt or compiled from source so I'm not sure where 32-bit programs might have leaked in from.

On another note, some people suggest adding -lz at the end of gcc/g++ commands. Can this flag perhaps be passed down to gcc/g++ via CMake?

List of "some people"

• https://github.com/madmonkey1907/hakchi/issues/48#issuecomment-376087430
• https://stackoverflow.com/a/2395371
• https://stackoverflow.com/a/30974901

[simonseo]

I should note, I'm a complete newb to CMake so I don't even know what I'm talking about right now. Appreciate your patience!

[C0DED00D]

I am not a CMake expert either. -lz should tell it to link in zlib (which CMake should already take care of), but it seems worth a shot. I believe if you add the following lines: SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -lz")

It'll add zlib explicitly to the command line. Can you give that a shot?

[simonseo]

I wasn't sure where to put the line so I tried putting it here and there but with no luck. Any suggestions on where to put it?

[C0DED00D]

I think at the top of the file would be the best - although I imagine it should work anywhere...

[simonseo]

Tried it and wasn't lucky this time either. I'll pause using this project for a bit but will come back around! Thanks for the help 🚀

[C0DED00D]

Sorry for the slow response. Have you tried building zlib from scratch and then changing the Cmake file to point to the version you built directly?

To do that, I'd delete this: find_package(ZLIB REQUIRED) if (NOT ZLIB_FOUND) message(FATAL_ERROR "Missing zlib libs.") endif (NOT ZLIB_FOUND)

and then set ZLIB_LIBRARIES to the directory of the zlib you built: set(ZLIB_LIBRARIES "path/to/zlib/you/built")

[simonseo]

Good news! It worked!

For future reference if anyone stumbles upon this thread:

I compiled zlib from source and replaced the find_package and if statements with set(ZLIB_LIBRARIES "/path/to/libz.so") as you told me to.

However, this wasn't enough to solve the issue. I also had to make the following change:

if (MSVC)
target_link_libraries(process
  common
-  "${ZLIB_LIBRARIES}"
   "${draco_LIBRARY_DIR}/dracodec.lib"
   "${giflib_LIBRARY_DIR}/giflib.lib"
   "${stb_image_LIBRARY_DIR}/stb_image.lib"
   "${PNG_LIBRARY}"
+  "${ZLIB_LIBRARIES}"
   "${TURBOJPEG_LIBRARY}"
 )
 else (MSVC)

 target_link_libraries(process
   common
-  "${ZLIB_LIBRARIES}"
   "${draco_LIBRARY_DIR}/libdracodec.a"
   "${giflib_LIBRARY_DIR}/libgiflib.a"
   "${stb_image_LIBRARY_DIR}/libstb_image.a"
   "${PNG_LIBRARY}"
+  "${ZLIB_LIBRARIES}"
  "${TURBOJPEG_LIBRARY}"
)
endif (MSVC)

I'm guessing it's because the PNG library depends on zlib so zlib had to be placed after the PNG library. Not sure why it wasn't a problem on my macOS though.

I'll submit a PR that fixes the library order in process/CMakeLists.txt and downloads zlib as a dependency in ufginstall.py if you think that'll be helpful.

[C0DED00D]

I'm so glad that worked for you! It'd be worth trying to download zlib as a dependency, but it seems strange that the CMake check for zlib was executing successfully in your case but it would fail to link.

[kowsheek]

Making these changes gives me the follow error still, assuming it can't find Zlib. Any help would be really appreciated @C0DED00D @simonseo!

PNG: Run: cmake /areo/areo-export-service/usd_from_gltf/ufg_build/src/libpng-1.6.37 -DCMAKE_INSTALL_PREFIX=/areo/areo-export-service/usd_from_gltf/ufg_build -DCMAKE_PREFIX_PATH=/areo/areo-export-service/usd_from_gltf/ufg_build -DCMAKE_POSITION_INDEPENDENT_CODE=1
-- The C compiler identification is GNU 7.4.0
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
CMake Error at /usr/share/cmake-3.10/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find ZLIB (missing: ZLIB_LIBRARY ZLIB_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/share/cmake-3.10/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.10/Modules/FindZLIB.cmake:112 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  CMakeLists.txt:42 (find_package)

[simonseo]

@kowsheek I’m not sure what the problem might be, but it would be helpful if you provide these information: Which OS are you on? Can you post how your usd_from_gltf/process/CMakeLists.txt looks like right now? Also, where did you place your libz.so file?

[kowsheek]

Thanks for responding @simonseo! My CMakeLists.txt file. I'm on Ubuntu:18.04. I'll be doing further investigation today and see where I get, happy to provide further information.

[simonseo]

@kowsheek I think CMake tries to set ZLIB_INCLUDE_DIR to where zlib.h is. Maybe you don't have that file? If you haven't, try sudo apt install zlib1g zlib1g-dev. zlib1g-dev includes zlib.h

[kowsheek]

It's the strangest thing, installing zlib1g-dev and changing the order of "${ZLIB_LIBRARIES}" fixed the ZLIB problem for me.

[C0DED00D]

Hmmm.. This is a bummer that zlib keeps causing issues. From the perspective of CMake, it appears we are doing the right thing, but it's not working out for everyone. Do you have any suggestions on how to improve this?

Thanks!

[kowsheek]

@C0DED00D thanks for responding! As you mentioned previously, the process could be smoothed over by including it in the installation process.

[amoravanszky]

I hacked around this same problem on windows as follows:

1) clone and build zlib from https://github.com/madler/zlib to C:/ZLIBINSTALLPATH/zlib 2) Edit YOURINSTALLPATH\usd_from_gltf\tools\ufginstall\ufginstall.py to add ++ lines:

if platform.system() == 'Windows':
  gen_args.append('-DCMAKE_GENERATOR_PLATFORM=x64')

++ gen_args.append('-DPNG_BUILD_ZLIB=yesplease') ++ gen_args.append('-DZLIB_INCLUDE_DIR=C:/ZLIBINSTALLPATH/zlib') ++ gen_args.append('-DZLIB_LIBRARY=C:/ZLIBINSTALLPATH/zlib/Release/zlib.lib') build_args.append('/m') # Multithreaded build.

Not the cleanest or most elegant but it works.

[C0DED00D]

Thanks for the info. I think we should try to download zlib as part of the python script and link against a local copy since it seems like this is problematic for a lot of folks. We'll add that to our backlog! Thanks for all of the investigations!

[hujasonx]

Hi all, thanks for investigating this - I believe this commit should fix these problems going into the future. I'll close this as most of you have already solved your issues, but feel free to reopen if I broke the build for anyone or it still hasn't been resolved.