search

fhanau / Efficient-Compression-Tool

Fast and effective C++ file optimizer
Apache License 2.0
596 stars 41 forks source link

build fail on ubuntu (google colab) #117

Closed eyaler closed 2 years ago

eyaler commented 2 years ago

I am not able to build on google colab (tried both master and v0.9.3 tag)

!git clone --recursive https://github.com/fhanau/Efficient-Compression-Tool
!apt -y install nasm
%cd Efficient-Compression-Tool
!mkdir build
%cd build
!cmake ../src
!make
Cloning into 'Efficient-Compression-Tool'...
remote: Enumerating objects: 5345, done.
remote: Counting objects: 100% (460/460), done.
remote: Compressing objects: 100% (170/170), done.
remote: Total 5345 (delta 314), reused 385 (delta 290), pack-reused 4885
Receiving objects: 100% (5345/5345), 4.80 MiB | 15.35 MiB/s, done.
Resolving deltas: 100% (3865/3865), done.
Submodule 'src/libpng' (https://github.com/glennrp/libpng.git) registered for path 'src/libpng'
Submodule 'src/mozjpeg' (https://github.com/fhanau/mozjpeg.git) registered for path 'src/mozjpeg'
Cloning into '/content/Efficient-Compression-Tool/src/libpng'...
remote: Enumerating objects: 107446, done.        
remote: Total 107446 (delta 0), reused 0 (delta 0), pack-reused 107446        
Receiving objects: 100% (107446/107446), 39.34 MiB | 19.01 MiB/s, done.
Resolving deltas: 100% (97252/97252), done.
Cloning into '/content/Efficient-Compression-Tool/src/mozjpeg'...
remote: Enumerating objects: 19583, done.        
remote: Counting objects: 100% (1945/1945), done.        
remote: Compressing objects: 100% (575/575), done.        
remote: Total 19583 (delta 1359), reused 1906 (delta 1344), pack-reused 17638
Receiving objects: 100% (19583/19583), 14.21 MiB | 21.53 MiB/s, done.
Resolving deltas: 100% (13924/13924), done.
Submodule path 'src/libpng': checked out '0a158f3506502dfa23edfc42790dfaed82efba17'
Submodule path 'src/mozjpeg': checked out 'cf6facaedb8f8ded44e6c41b0db8a721d329ff22'
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  libnvidia-common-460
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
  nasm
0 upgraded, 1 newly installed, 0 to remove and 4 not upgraded.
Need to get 359 kB of archives.
After this operation, 2,831 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 nasm amd64 2.13.02-0.1 [359 kB]
Fetched 359 kB in 0s (1,529 kB/s)
Selecting previously unselected package nasm.
(Reading database ... 123942 files and directories currently installed.)
Preparing to unpack .../nasm_2.13.02-0.1_amd64.deb ...
Unpacking nasm (2.13.02-0.1) ...
Setting up nasm (2.13.02-0.1) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
/content/Efficient-Compression-Tool
/content/Efficient-Compression-Tool/build
-- The CXX compiler identification is GNU 7.5.0
-- The C compiler identification is GNU 7.5.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Performing Test HAVE_LD_VERSION_SCRIPT
-- Performing Test HAVE_LD_VERSION_SCRIPT - Success
-- Symbol prefix: 
CMake Warning (dev) at mozjpeg/CMakeLists.txt:12 (project):
  Policy CMP0048 is not set: project() command manages VERSION variables.
  Run "cmake --help-policy CMP0048" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.

  The following variable(s) would be set to empty:

    PROJECT_VERSION
    PROJECT_VERSION_MAJOR
    PROJECT_VERSION_MINOR
    PROJECT_VERSION_PATCH
This warning is for project developers.  Use -Wno-dev to suppress it.

-- CMAKE_BUILD_TYPE = Release
-- VERSION = 4.1.2, BUILD = 20221102
-- 64-bit build (x86_64)
-- CMAKE_INSTALL_PREFIX = /usr/local
CMake Warning (dev) at mozjpeg/cmakescripts/GNUInstallDirs.cmake:175 (elseif):
  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  Quoted variables like "var" will no longer be dereferenced when the policy
  is set to NEW.  Since the policy is not set the OLD behavior will be used.
Call Stack (most recent call first):
  mozjpeg/cmakescripts/GNUInstallDirs.cmake:233 (GNUInstallDirs_set_install_dir)
  mozjpeg/CMakeLists.txt:152 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- CMAKE_INSTALL_BINDIR = bin (/usr/local/bin)
-- CMAKE_INSTALL_DATAROOTDIR = share (/usr/local/share)
-- CMAKE_INSTALL_DOCDIR = share/doc/ect (/usr/local/share/doc/ect)
-- CMAKE_INSTALL_INCLUDEDIR = include (/usr/local/include)
-- CMAKE_INSTALL_LIBDIR = lib (/usr/local/lib)
-- CMAKE_INSTALL_MANDIR = share/man (/usr/local/share/man)
-- Shared libraries disabled (ENABLE_SHARED = 0)
-- Static libraries enabled (ENABLE_STATIC = 1)
-- 12-bit JPEG support disabled (WITH_12BIT = 0)
-- Arithmetic decoding support enabled (WITH_ARITH_DEC = 1)
-- Arithmetic encoding support enabled (WITH_ARITH_ENC = 1)
-- TurboJPEG API library disabled (WITH_TURBOJPEG = 0)
-- TurboJPEG Java wrapper disabled (WITH_JAVA = 0)
-- In-memory source/destination managers enabled (WITH_MEM_SRCDST = 1)
-- Emulating libjpeg API/ABI v6.2 (WITH_JPEG7 = 0, WITH_JPEG8 = 0)
-- libjpeg API shared library version = 62.3.0
-- Compiler flags =  -Wall -Wno-sign-compare -Wno-unused -Wno-unused-parameter -mpclmul -msse4.2 -Ofast -DNDEBUG
-- Linker flags =  
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of size_t
-- Check size of size_t - done
-- Check size of unsigned long
-- Check size of unsigned long - done
-- Performing Test HAVE_BUILTIN_CTZL
-- Performing Test HAVE_BUILTIN_CTZL - Success
-- Performing Test RIGHT_SHIFT_IS_UNSIGNED
-- Performing Test RIGHT_SHIFT_IS_UNSIGNED - Failed
-- Performing Test INLINE_WORKS
-- Performing Test INLINE_WORKS - Success
-- INLINE = __inline__ __attribute__((always_inline)) (FORCE_INLINE = 1)
-- Performing Test HAVE_VERSION_SCRIPT
-- Performing Test HAVE_VERSION_SCRIPT - Success
-- Linker supports GNU-style version scripts
-- CMAKE_EXECUTABLE_SUFFIX = 
-- Looking for a ASM_NASM compiler
-- Looking for a ASM_NASM compiler - /usr/bin/nasm
-- The ASM_NASM compiler identification is NASM
-- Found assembler: /usr/bin/nasm
-- CMAKE_ASM_NASM_COMPILER = /usr/bin/nasm
-- CMAKE_ASM_NASM_OBJECT_FORMAT = elf64
-- CMAKE_ASM_NASM_FLAGS =  -DELF -D__x86_64__ 
-- SIMD extensions: x86_64 (WITH_SIMD = 1)
-- FLOATTEST = sse
-- RPM architecture = x86_64, DEB architecture = amd64
-- Configuring done
-- Generating done
-- Build files have been written to: /content/Efficient-Compression-Tool/build
[  0%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jsimdcpu.asm.o
[  1%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jfdctflt-sse.asm.o
[  1%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jccolor-sse2.asm.o
[  2%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jcgray-sse2.asm.o
[  2%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jchuff-sse2.asm.o
[  2%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jcphuff-sse2.asm.o
[  4%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jcsample-sse2.asm.o
[  4%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jdcolor-sse2.asm.o
[  5%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jdmerge-sse2.asm.o
[  5%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jdsample-sse2.asm.o
[  5%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jfdctfst-sse2.asm.o
[  6%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jfdctint-sse2.asm.o
[  6%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jidctflt-sse2.asm.o
[  8%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jidctfst-sse2.asm.o
[  8%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jidctint-sse2.asm.o
[  8%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jidctred-sse2.asm.o
[  9%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jquantf-sse2.asm.o
[  9%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jquanti-sse2.asm.o
[  9%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jccolor-avx2.asm.o
[ 10%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jcgray-avx2.asm.o
[ 10%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jcsample-avx2.asm.o
[ 12%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jdcolor-avx2.asm.o
[ 12%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jdmerge-avx2.asm.o
[ 12%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jdsample-avx2.asm.o
[ 13%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jfdctint-avx2.asm.o
[ 13%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jidctint-avx2.asm.o
[ 15%] Building ASM_NASM object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jquanti-avx2.asm.o
[ 15%] Building C object mozjpeg/simd/CMakeFiles/simd.dir/x86_64/jsimd.c.o
[ 15%] Built target simd
[ 16%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcapimin.c.o
[ 16%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcapistd.c.o
[ 17%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jccoefct.c.o
[ 17%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jccolor.c.o
[ 17%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcdctmgr.c.o
[ 19%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jchuff.c.o
[ 19%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcext.c.o
[ 20%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcicc.c.o
[ 20%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcinit.c.o
[ 20%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcmainct.c.o
[ 21%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcmarker.c.o
[ 21%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcmaster.c.o
[ 23%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcomapi.c.o
[ 23%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcparam.c.o
[ 23%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcphuff.c.o
[ 24%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcprepct.c.o
[ 24%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcsample.c.o
[ 24%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jctrans.c.o
[ 26%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdapimin.c.o
[ 26%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdapistd.c.o
[ 27%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdatadst.c.o
[ 27%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdatasrc.c.o
[ 27%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdcoefct.c.o
[ 28%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdcolor.c.o
[ 28%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jddctmgr.c.o
[ 30%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdhuff.c.o
[ 30%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdicc.c.o
[ 30%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdinput.c.o
[ 31%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdmainct.c.o
[ 31%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdmarker.c.o
[ 32%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdmaster.c.o
[ 32%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdmerge.c.o
[ 32%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdphuff.c.o
[ 34%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdpostct.c.o
[ 34%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdsample.c.o
[ 34%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdtrans.c.o
[ 35%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jerror.c.o
[ 35%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jfdctflt.c.o
[ 36%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jfdctfst.c.o
[ 36%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jfdctint.c.o
[ 36%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jidctflt.c.o
[ 38%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jidctfst.c.o
[ 38%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jidctint.c.o
[ 39%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jidctred.c.o
[ 39%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jquant1.c.o
[ 39%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jquant2.c.o
[ 41%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jutils.c.o
[ 41%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jmemmgr.c.o
[ 42%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jmemnobs.c.o
[ 42%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jaricom.c.o
[ 42%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jcarith.c.o
[ 43%] Building C object mozjpeg/CMakeFiles/jpeg-static.dir/jdarith.c.o
[ 43%] Linking C static library libjpeg.a
[ 43%] Built target jpeg-static
[ 43%] Building CXX object leanify/CMakeFiles/leanify.dir/leanify.cpp.o
In file included from /content/Efficient-Compression-Tool/src/leanify/zip.h:4:0,
                 from /content/Efficient-Compression-Tool/src/leanify/leanify.cpp:14:
/content/Efficient-Compression-Tool/src/leanify/../main.h:17:10: fatal error: filesystem: No such file or directory
 #include <filesystem>
          ^~~~~~~~~~~~
compilation terminated.
leanify/CMakeFiles/leanify.dir/build.make:75: recipe for target 'leanify/CMakeFiles/leanify.dir/leanify.cpp.o' failed
make[2]: *** [leanify/CMakeFiles/leanify.dir/leanify.cpp.o] Error 1
CMakeFiles/Makefile2:330: recipe for target 'leanify/CMakeFiles/leanify.dir/all' failed
make[1]: *** [leanify/CMakeFiles/leanify.dir/all] Error 2
Makefile:135: recipe for target 'all' failed
make: *** [all] Error 2
fhanau commented 2 years ago

ECT uses std::filesystem from the C++ standard library. This is supported since C++ 17. Try to use a compiler that has full support of the C++17 features. For GCC, this should be supported fully starting with version 9, so since a few releases.

eyaler commented 2 years ago

i was able to build ect on colab after upgrading gcc with this script: https://gist.github.com/alexandreelise/192a63e287018ddfc896bbcb70b219d4

also, to use the file on a fresh colab you need to install GLIBCXX_3.4.26:

!add-apt-repository -y ppa:ubuntu-toolchain-r/test
!apt upgrade libstdc++6

@fhanau is there an option to create a standalone linux build which includes this library?