intel / intel-graphics-compiler

Other
600 stars 155 forks source link

Huge size of conda-forge based binaries #341

Open ZzEeKkAa opened 1 day ago

ZzEeKkAa commented 1 day ago

Hello there!

I'm trying to recover IGC build in conda-forge, however faced few issues.

~First of all I've been able to compile the package locally, but on the CI it fails with this kind of errors:~

2024-10-01T05:39:13.0889269Z objdump: $SRC_DIR/build/IGC/CMakeFiles/igc_dll.dir/./AdaptorCommon/RayTracing/OverrideTMaxPass.cpp.o: unable to initialize decompress status for section .debug_info
2024-10-01T05:39:13.0889818Z objdump: $SRC_DIR/build/IGC/CMakeFiles/igc_dll.dir/./AdaptorCommon/RayTracing/OverrideTMaxPass.cpp.o: File format not recognized

~Build script: https://github.com/ZzEeKkAa/intel-graphics-compiler-feedstock/blob/update/version_1.0.17537.20/recipe/build.sh~ ~Error log: https://dev.azure.com/conda-forge/84710dde-1620-425b-80d0-4cf5baca359d/_apis/build/builds/1042113/logs/7~

Another issue is that resulted size of libigc.so is much bigger compare to attached debian packages attached to the release:

Debian package:

$ du -sh core/usr/local/lib/libigc*
0       core/usr/local/lib/libigc.so
0       core/usr/local/lib/libigc.so.1
83M     core/usr/local/lib/libigc.so.1.0.17537.20
$ ldd core/usr/local/lib/libigc.so
        linux-vdso.so.1 (0x00007fff701aa000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f46cb65c000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f46cb575000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f46cb555000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f46cb32c000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f46d0cb0000)

Conda build:

$ du -sh $CONDA_PREFIX/lib/libigc*
0       /home/yevhenii/.miniforge3/envs/pti-test/lib/libigc.so
0       /home/yevhenii/.miniforge3/envs/pti-test/lib/libigc.so.1
795M    /home/yevhenii/.miniforge3/envs/pti-test/lib/libigc.so.1.0.1
$ ldd $CONDA_PREFIX/lib/libigc.so
        linux-vdso.so.1 (0x00007ffec9ded000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f3669c6a000)
        libLLVMSPIRVLib.so.14 => /home/yevhenii/.miniforge3/envs/pti-test/lib/./libLLVMSPIRVLib.so.14 (0x00007f3669834000)
        libSPIRV-Tools.so => /home/yevhenii/.miniforge3/envs/pti-test/lib/./libSPIRV-Tools.so (0x00007f366969b000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3669696000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3669691000)
        libLLVM-14.so => /home/yevhenii/.miniforge3/envs/pti-test/lib/./libLLVM-14.so (0x00007f3662633000)
        libz.so.1 => /home/yevhenii/.miniforge3/envs/pti-test/lib/./libz.so.1 (0x00007f3662618000)
        libstdc++.so.6 => /home/yevhenii/.miniforge3/envs/pti-test/lib/./libstdc++.so.6 (0x00007f3662435000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f366234e000)
        libgcc_s.so.1 => /home/yevhenii/.miniforge3/envs/pti-test/lib/./libgcc_s.so.1 (0x00007f366232f000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3662106000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f366cc8f000)

Do you know why is the package is so huge in my build, despite the fact that I'm dynamically linking towards spirv-tools/llvm-spirv and all others libraries? How can I approach this issue to understand where the size is coming from? Maybe it builds with debug symbols by default?

The pending PR could be found here: https://github.com/conda-forge/intel-graphics-compiler-feedstock/pull/58

isuruf commented 1 day ago

Running strip on libigc.so reduces the size a lot.

ZzEeKkAa commented 1 day ago

@isuruf Indeed!

This is before and after:

47M     /home/yevhenii/.miniforge3/envs/pti-test/lib/libigc.so.1.0.1
795M    /home/yevhenii/.miniforge3/envs/pti-test/lib/libigc.so.1.0.1.bkp

I'm wondering if strip was called during debian package build. How was debug package generated? Understanding how to achieve this, will help to create similar split in conda-forge.

ZzEeKkAa commented 1 day ago

And another question is why Release target produced unstriped binary.

isuruf commented 1 day ago

https://github.com/intel/intel-graphics-compiler/blob/c9396878a1d372875745262c9d02e82bcc6352ba/IGC/CMakeLists.txt#L1127-L1139

ZzEeKkAa commented 1 day ago

Thank you @isuruf !