search

NVIDIA / nvImageCodec

A nvImageCodec library of GPU- and CPU- accelerated codecs featuring a unified interface
https://docs.nvidia.com/cuda/nvimagecodec/index.html
Apache License 2.0
53 stars 4 forks source link

Building from source fails when `nvjpeg2k` is enabled #7

Open mantasu opened 1 month ago

mantasu commented 1 month ago

Version

0.2.0

Describe the bug.

When trying to build from source using the most recent nvjpeg2k version (0.8.0 currently), make fails halfway through.

I believe it is because the header file which defines struct nvjpeg2kEncodeConfig_t might have changed with the newer version, meaning enable_custom_precincts was changed to num_precincts_init (I could be wrong):

typedef struct 
{
    ...
    uint32_t num_precincts_init; // to enable, set to a non zero value corresponding to the no of valid entries in precinct_width and precinct_height
    ...
} nvjpeg2kEncodeConfig_t;

Minimum reproducible example

Simply follow the instructions in README.md for building from source:

git lfs clone https://github.com/NVIDIA/nvImageCodec.git
cd nvimagecodec
git submodule update --init --recursive --depth 1
mkdir build
cd build
export CUDACXX=nvcc
cmake .. -DCMAKE_BUILD_TYPE=Release
make

Environment details

Click here to see environment details

     **git***
tools/print_env.sh: 24: [: true: unexpected operator
     Not inside a git repository

     ***OS Information***
     DISTRIB_ID=Ubuntu
     DISTRIB_RELEASE=22.04
     DISTRIB_CODENAME=jammy
     DISTRIB_DESCRIPTION="Ubuntu 22.04.4 LTS"
     PRETTY_NAME="Ubuntu 22.04.4 LTS"
     NAME="Ubuntu"
     VERSION_ID="22.04"
     VERSION="22.04.4 LTS (Jammy Jellyfish)"
     VERSION_CODENAME=jammy
     ID=ubuntu
     ID_LIKE=debian
     HOME_URL="https://www.ubuntu.com/"
     SUPPORT_URL="https://help.ubuntu.com/"
     BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
     PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
     UBUNTU_CODENAME=jammy
     Linux MSI 5.15.146.1-microsoft-standard-WSL2 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

     ***GPU Information***
     Sat Jun  1 11:24:48 2024
     +---------------------------------------------------------------------------------------+
     | NVIDIA-SMI 545.23.08              Driver Version: 546.33       CUDA Version: 12.3     |
     |-----------------------------------------+----------------------+----------------------+
     | GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
     |                                         |                      |               MIG M. |
     |=========================================+======================+======================|
     |   0  NVIDIA GeForce RTX 3080 ...    On  | 00000000:01:00.0  On |                  N/A |
     | N/A   46C    P8              11W / 155W |    388MiB / 16384MiB |      0%      Default |
     |                                         |                      |                  N/A |
     +-----------------------------------------+----------------------+----------------------+

     +---------------------------------------------------------------------------------------+
     | Processes:                                                                            |
     |  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
     |        ID   ID                                                             Usage      |
     |=======================================================================================|
     |    0   N/A  N/A     61831      C   /python3.12                               N/A      |
     +---------------------------------------------------------------------------------------+

     ***CPU***
     Architecture:                       x86_64
     CPU op-mode(s):                     32-bit, 64-bit
     Address sizes:                      39 bits physical, 48 bits virtual
     Byte Order:                         Little Endian
     CPU(s):                             24
     On-line CPU(s) list:                0-23
     Vendor ID:                          GenuineIntel
     Model name:                         12th Gen Intel(R) Core(TM) i7-12800HX
     CPU family:                         6
     Model:                              151
     Thread(s) per core:                 2
     Core(s) per socket:                 12
     Socket(s):                          1
     Stepping:                           2
     BogoMIPS:                           4607.99
     Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves avx_vnni umip waitpkg gfni vaes vpclmulqdq rdpid movdiri movdir64b fsrm md_clear serialize flush_l1d arch_capabilities
     Virtualization:                     VT-x
     Hypervisor vendor:                  Microsoft
     Virtualization type:                full
     L1d cache:                          576 KiB (12 instances)
     L1i cache:                          384 KiB (12 instances)
     L2 cache:                           15 MiB (12 instances)
     L3 cache:                           25 MiB (1 instance)
     Vulnerability Gather data sampling: Not affected
     Vulnerability Itlb multihit:        Not affected
     Vulnerability L1tf:                 Not affected
     Vulnerability Mds:                  Not affected
     Vulnerability Meltdown:             Not affected
     Vulnerability Mmio stale data:      Not affected
     Vulnerability Retbleed:             Mitigation; Enhanced IBRS
     Vulnerability Spec rstack overflow: Not affected
     Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl and seccomp
     Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
     Vulnerability Spectre v2:           Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
     Vulnerability Srbds:                Not affected
     Vulnerability Tsx async abort:      Not affected

     ***CMake***
     /usr/bin/cmake
     cmake version 3.22.1

     CMake suite maintained and supported by Kitware (kitware.com/cmake).

     ***g++***
     /usr/bin/g++
     g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
     Copyright (C) 2021 Free Software Foundation, Inc.
     This is free software; see the source for copying conditions.  There is NO
     warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

     ***nvcc***
     /usr/local/cuda/bin/nvcc
     nvcc: NVIDIA (R) Cuda compiler driver
     Copyright (c) 2005-2023 NVIDIA Corporation
     Built on Wed_Nov_22_10:17:15_PST_2023
     Cuda compilation tools, release 12.3, V12.3.107
     Build cuda_12.3.r12.3/compiler.33567101_0

     ***Python***
     /home/mantasu/programs/anaconda/envs/faceget/bin/python
     Python 3.12.3

     ***Environment Variables***
     PATH                            : /home/mantasu/programs/anaconda/envs/faceget/bin:/home/mantasu/.vscode-server/bin/dc96b837cf6bb4af9cd736aa3af08cf8279f7685/bin/remote-cli:/home/mantasu/programs/anaconda/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/Program
     Files/Common                    : Files/Oracle/Java/javapath:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0:/mnt/c/WINDOWS/System32/OpenSSH:/mnt/c/Program
     Files                           : (x86)/NVIDIA
     Corporation/PhysX/Common:/mnt/c/Program: Files/NVIDIA
     Corporation/NVIDIA              : NvDLISR:/mnt/c/Program
     Files/dotnet:/mnt/c/Users/mantasu/AppData/Local/Microsoft/WindowsApps:/mnt/c/Programs/Microsoft: VS
     Code/bin:/snap/bin:/usr/local/cuda/bin:/home/mantasu/.local/share/coursier/bin:/home/mantasu/programs/anaconda/envs/faceget/bin:/home/mantasu/.vscode-server/bin/dc96b837cf6bb4af9cd736aa3af08cf8279f7685/bin/remote-cli:/home/mantasu/programs/anaconda/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/Program: Files/Common
     Files/Oracle/Java/javapath:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0:/mnt/c/WINDOWS/System32/OpenSSH:/mnt/c/Program: Files
     (x86)/NVIDIA                    : Corporation/PhysX/Common:/mnt/c/Program
     Files/NVIDIA                    : Corporation/NVIDIA
     NvDLISR:/mnt/c/Program          : Files/dotnet:/mnt/c/Users/mantasu/AppData/Local/Microsoft/WindowsApps:/mnt/c/Programs/Microsoft
     VS                              : Code/bin:/snap/bin:/usr/local/cuda/bin:/home/mantasu/.local/share/coursier/bin:/home/mantasu/.vscode-server/bin/dc96b837cf6bb4af9cd736aa3af08cf8279f7685/bin/remote-cli:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/Program
     Files/Common                    : Files/Oracle/Java/javapath:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0:/mnt/c/WINDOWS/System32/OpenSSH:/mnt/c/Program
     Files                           : (x86)/NVIDIA
     Corporation/PhysX/Common:/mnt/c/Program: Files/NVIDIA
     Corporation/NVIDIA              : NvDLISR:/mnt/c/Program
     Files/dotnet:/mnt/c/Users/mantasu/AppData/Local/Microsoft/WindowsApps:/mnt/c/Programs/Microsoft: VS
     Code/bin                        :
     LD_LIBRARY_PATH                 :
     NUMBAPRO_NVVM                   :
     NUMBAPRO_LIBDEVICE              :
     CONDA_PREFIX                    : /home/mantasu/programs/anaconda/envs/faceget
     PYTHON_PATH                     :

     ***conda packages***
     conda is /home/mantasu/programs/anaconda/condabin/conda
     /home/mantasu/programs/anaconda/condabin/conda
     # packages in environment at /home/mantasu/programs/anaconda/envs/faceget:
     #
     # Name                    Version                   Build  Channel
     _libgcc_mutex             0.1                 conda_forge    conda-forge
     _openmp_mutex             4.5                       2_gnu    conda-forge
     absl-py                   2.1.0                    pypi_0    pypi
     appdirs                   1.4.4                    pypi_0    pypi
     asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
     astunparse                1.6.3                    pypi_0    pypi
     blas                      2.16                        mkl    conda-forge
     bzip2                     1.0.8                hd590300_5    conda-forge
     ca-certificates           2024.2.2             hbcca054_0    conda-forge
     certifi                   2024.2.2                 pypi_0    pypi
     charset-normalizer        3.3.2                    pypi_0    pypi
     comm                      0.2.2              pyhd8ed1ab_0    conda-forge
     cuda-cudart               12.1.105                      0    nvidia
     cuda-cupti                12.1.105                      0    nvidia
     cuda-libraries            12.1.0                        0    nvidia
     cuda-nvrtc                12.1.105                      0    nvidia
     cuda-nvtx                 12.1.105                      0    nvidia
     cuda-opencl               12.4.127                      0    nvidia
     cuda-runtime              12.1.0                        0    nvidia
     cvcuda-cu12               0.8.0b0                  pypi_0    pypi
     debugpy                   1.8.1           py312h30efb56_0    conda-forge
     decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
     exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
     executing                 2.0.1              pyhd8ed1ab_0    conda-forge
     filelock                  3.14.0             pyhd8ed1ab_0    conda-forge
     flatbuffers               24.3.25                  pypi_0    pypi
     gast                      0.5.4                    pypi_0    pypi
     gmp                       6.3.0                h59595ed_1    conda-forge
     gmpy2                     2.1.5           py312h1d5cde6_1    conda-forge
     google-pasta              0.2.0                    pypi_0    pypi
     grpcio                    1.64.0                   pypi_0    pypi
     h5py                      3.11.0                   pypi_0    pypi
     idna                      3.7                      pypi_0    pypi
     importlib-metadata        7.1.0              pyha770c72_0    conda-forge
     importlib_metadata        7.1.0                hd8ed1ab_0    conda-forge
     intel-openmp              2023.1.0         hdb19cb5_46306
     ipykernel                 6.29.3             pyhd33586a_0    conda-forge
     ipython                   8.25.0             pyh707e725_0    conda-forge
     jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
     jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
     jupyter_client            8.6.2              pyhd8ed1ab_0    conda-forge
     jupyter_core              5.7.2           py312h7900ff3_0    conda-forge
     keras                     3.3.3                    pypi_0    pypi
     keyutils                  1.6.1                h166bdaf_0    conda-forge
     krb5                      1.21.2               h659d440_0    conda-forge
     ld_impl_linux-64          2.40                 hf3520f5_1    conda-forge
     libblas                   3.8.0                    16_mkl    conda-forge
     libcblas                  3.8.0                    16_mkl    conda-forge
     libclang                  18.1.1                   pypi_0    pypi
     libcublas                 12.1.0.26                     0    nvidia
     libcufft                  11.0.2.4                      0    nvidia
     libcufile                 1.9.1.3                       0    nvidia
     libcurand                 10.3.5.147                    0    nvidia
     libcusolver               11.4.4.55                     0    nvidia
     libcusparse               12.0.2.55                     0    nvidia
     libedit                   3.1.20191231         he28a2e2_2    conda-forge
     libexpat                  2.6.2                h59595ed_0    conda-forge
     libffi                    3.4.2                h7f98852_5    conda-forge
     libgcc-ng                 13.2.0               h77fa898_7    conda-forge
     libgfortran-ng            7.5.0               h14aa051_20    conda-forge
     libgfortran4              7.5.0               h14aa051_20    conda-forge
     libgomp                   13.2.0               h77fa898_7    conda-forge
     liblapack                 3.8.0                    16_mkl    conda-forge
     liblapacke                3.8.0                    16_mkl    conda-forge
     libnpp                    12.0.2.50                     0    nvidia
     libnsl                    2.0.1                hd590300_0    conda-forge
     libnvjitlink              12.1.105                      0    nvidia
     libnvjpeg                 12.1.1.14                     0    nvidia
     libsodium                 1.0.18               h36c2ea0_1    conda-forge
     libsqlite                 3.45.3               h2797004_0    conda-forge
     libstdcxx-ng              13.2.0               hc0a3c3a_7    conda-forge
     libuuid                   2.38.1               h0b41bf4_0    conda-forge
     libxcrypt                 4.4.36               hd590300_1    conda-forge
     libzlib                   1.2.13               h4ab18f5_6    conda-forge
     llvm-openmp               15.0.7               h0cdce71_0    conda-forge
     mako                      1.3.5                    pypi_0    pypi
     markdown                  3.6                      pypi_0    pypi
     markdown-it-py            3.0.0                    pypi_0    pypi
     markupsafe                2.1.5           py312h98912ed_0    conda-forge
     matplotlib-inline         0.1.7              pyhd8ed1ab_0    conda-forge
     mdurl                     0.1.2                    pypi_0    pypi
     mkl                       2020.2                      256
     ml-dtypes                 0.3.2                    pypi_0    pypi
     mpc                       1.3.1                hfe3b2da_0    conda-forge
     mpfr                      4.2.1                h9458935_1    conda-forge
     mpmath                    1.3.0              pyhd8ed1ab_0    conda-forge
     namex                     0.0.8                    pypi_0    pypi
     ncurses                   6.5                  h59595ed_0    conda-forge
     nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
     networkx                  3.3                pyhd8ed1ab_1    conda-forge
     numpy                     1.26.4                   pypi_0    pypi
     nvidia-cublas-cu12        12.3.4.1                 pypi_0    pypi
     nvidia-cuda-cupti-cu12    12.3.101                 pypi_0    pypi
     nvidia-cuda-nvcc-cu12     12.3.107                 pypi_0    pypi
     nvidia-cuda-nvrtc-cu12    12.3.107                 pypi_0    pypi
     nvidia-cuda-runtime-cu12  12.3.101                 pypi_0    pypi
     nvidia-cudnn-cu12         8.9.7.29                 pypi_0    pypi
     nvidia-cufft-cu12         11.0.12.1                pypi_0    pypi
     nvidia-curand-cu12        10.3.4.107               pypi_0    pypi
     nvidia-cusolver-cu12      11.5.4.101               pypi_0    pypi
     nvidia-cusparse-cu12      12.2.0.103               pypi_0    pypi
     nvidia-nccl-cu12          2.19.3                   pypi_0    pypi
     nvidia-nvimgcodec-cu12    0.2.0.0                  pypi_0    pypi
     nvidia-nvjitlink-cu12     12.3.101                 pypi_0    pypi
     opencv-python             4.9.0.80                 pypi_0    pypi
     openssl                   3.3.0                h4ab18f5_3    conda-forge
     opt-einsum                3.3.0                    pypi_0    pypi
     optree                    0.11.0                   pypi_0    pypi
     packaging                 24.0               pyhd8ed1ab_0    conda-forge
     parso                     0.8.4              pyhd8ed1ab_0    conda-forge
     pexpect                   4.9.0              pyhd8ed1ab_0    conda-forge
     pickleshare               0.7.5                   py_1003    conda-forge
     pillow                    10.3.0                   pypi_0    pypi
     pip                       24.0               pyhd8ed1ab_0    conda-forge
     platformdirs              4.2.2              pyhd8ed1ab_0    conda-forge
     prompt-toolkit            3.0.42             pyha770c72_0    conda-forge
     protobuf                  4.25.3                   pypi_0    pypi
     psutil                    5.9.8           py312h98912ed_0    conda-forge
     ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
     pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
     pycuda                    2024.1                   pypi_0    pypi
     pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
     python                    3.12.3          hab00c5b_0_cpython    conda-forge
     python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
     python_abi                3.12                    4_cp312    conda-forge
     pytools                   2024.1.3                 pypi_0    pypi
     pytorch                   2.3.0           py3.12_cuda12.1_cudnn8.9.2_0    pytorch
     pytorch-cuda              12.1                 ha16c6d3_5    pytorch
     pytorch-mutex             1.0                        cuda    pytorch
     pyyaml                    6.0.1           py312h98912ed_1    conda-forge
     pyzmq                     26.0.3          py312h8fd38d8_0    conda-forge
     readline                  8.2                  h8228510_1    conda-forge
     requests                  2.32.3                   pypi_0    pypi
     rich                      13.7.1                   pypi_0    pypi
     setuptools                70.0.0             pyhd8ed1ab_0    conda-forge
     six                       1.16.0             pyh6c4a22f_0    conda-forge
     stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
     sympy                     1.12            pypyh9d50eac_103    conda-forge
     tensorboard               2.16.2                   pypi_0    pypi
     tensorboard-data-server   0.7.2                    pypi_0    pypi
     tensorflow                2.16.1                   pypi_0    pypi
     termcolor                 2.4.0                    pypi_0    pypi
     tk                        8.6.13          noxft_h4845f30_101    conda-forge
     tornado                   6.4             py312h98912ed_0    conda-forge
     traitlets                 5.14.3             pyhd8ed1ab_0    conda-forge
     typing_extensions         4.11.0             pyha770c72_0    conda-forge
     tzdata                    2024a                h0c530f3_0    conda-forge
     urllib3                   2.2.1                    pypi_0    pypi
     wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
     werkzeug                  3.0.3                    pypi_0    pypi
     wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
     wrapt                     1.16.0                   pypi_0    pypi
     xz                        5.2.6                h166bdaf_0    conda-forge
     yaml                      0.2.5                h7f98852_2    conda-forge
     zeromq                    4.3.5                h75354e8_4    conda-forge
     zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
     zlib                      1.2.13               h4ab18f5_6    conda-forge

Relevant log output

[ 51%] Building CXX object extensions/nvjpeg2k/CMakeFiles/nvjpeg2k_ext.dir/nvjpeg2k_ext.cpp.o
[ 52%] Building CXX object extensions/nvjpeg2k/CMakeFiles/nvjpeg2k_ext.dir/cuda_encoder.cpp.o
nvImageCodec/extensions/nvjpeg2k/cuda_encoder.cpp: In lambda function:
nvImageCodec/extensions/nvjpeg2k/cuda_encoder.cpp:449:31: error: ‘struct nvjpeg2kEncodeConfig_t’ has no member named ‘enable_custom_precincts’
  449 |                 encode_config.enable_custom_precincts = 0;
      |                               ^~~~~~~~~~~~~~~~~~~~~~~
make[2]: *** [extensions/nvjpeg2k/CMakeFiles/nvjpeg2k_ext.dir/build.make:90: extensions/nvjpeg2k/CMakeFiles/nvjpeg2k_ext.dir/cuda_encoder.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:929: extensions/nvjpeg2k/CMakeFiles/nvjpeg2k_ext.dir/all] Error 2
make: *** [Makefile:166: all] Error 2

Other/Misc.

Package: nvjpeg2k
Version: 0.8.0.38-1
Priority: optional
Section: multiverse/devel
Source: libnvjpeg-2k
Maintainer: cudatools <cudatools@nvidia.com>
Installed-Size: 27.6 kB
Depends: nvjpeg2k0 (>= 0.8.0.38)
Download-Size: 9544 B
APT-Manual-Installed: yes
APT-Sources: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
Description: NVIDIA nvJPEG 2000
 NVIDIA nvJPEG 2000

Check for duplicates

jantonguirao commented 1 month ago

Thank you, @mantasu, for reporting this. Yes, nvjpeg2k changed its API in 0.8.0. The next release will have a fix for it, so that we will conditionally compile one or the other depending on the available nvjpeg2k version. We'll let you know when the release is out.