search

pthom / imgui_bundle

Your fast track to powerful GUIs: Dear ImGui Bundle, an extensive toolkit for Python and C++ with immediate mode efficiency.
https://pthom.github.io/imgui_bundle/
MIT License
701 stars 72 forks source link

_BrotliDecoderDecompress missing on macOS 14.5 intel #261

Closed virtuald closed 1 month ago

virtuald commented 1 month ago 
ImportError: dlopen(... site-packages/imgui_bundle/_imgui_bundle.cpython-312-darwin.so, 0x0002): symbol not found in flat namespace '_BrotliDecoderDecompress'

Works fine on my M3 mac, but not on the intel mac. Any thoughts on what's missing?

pthom commented 1 month ago

freetype is used by imgui to improve font rendering. And freetype uses Brotli.

You might try to compile from source and comment out those lines: https://github.com/pthom/imgui_bundle/blob/f8db311b9c0828330e6b277238243ae3ccb8db76/CMakeLists.txt#L167-L168

virtuald commented 1 month ago

Hm. But isn't all of that in the wheel? Or is it expecting something to be installed at runtime (maybe via brew?)

virtuald commented 1 month ago

Ok, so https://files.pythonhosted.org/packages/29/1b/155fb7bc2605e2e8d8134befa48870f24252d0010931b429c0a45116195e/imgui_bundle-1.5.2-cp312-cp312-macosx_14_0_x86_64.whl does not have .dylibs, but https://files.pythonhosted.org/packages/87/f7/e0c7f1461743fee24373156aa134f6822881bab6794807992b32916dbaa3/imgui_bundle-1.5.2-cp312-cp312-macosx_14_0_arm64.whl does. So it's something about the build?

pthom commented 1 month ago

Hi Dustin,

Building Python libraries with native linked libraries is always complex. Below is an analysis of what I see.

Main CMakeLists:

in the main CMakeLists of imgui_bundle, we call ibd_force_freetype_static_for_python()

https://github.com/pthom/imgui_bundle/blob/f8db311b9c0828330e6b277238243ae3ccb8db76/CMakeLists.txt#L165-L168

 ibd_check_freetype_availability(freetype_default) 
 # Note: to reduce wasm size, you may want to disable freetype on emscripten. 
 option(HELLOIMGUI_USE_FREETYPE "Use freetype for text rendering" ${freetype_default}) 
 ibd_force_freetype_static_for_python()

ibd_check_freetype_availability & ibd_force_freetype_static_for_python

This sets HELLOIMGUI_FREETYPE_STATIC=ON

https://github.com/pthom/imgui_bundle/blob/f8db311b9c0828330e6b277238243ae3ccb8db76/imgui_bundle_cmake/imgui_bundle_build_lib.cmake#L27-L50

function(ibd_force_freetype_static_for_python)
    # For python bindings, we force the usage of a static version of freetype
    if(IMGUI_BUNDLE_BUILD_PYTHON AND HELLOIMGUI_USE_FREETYPE)
        # if using vcpkg, we suppose it uses the static version of freetype
        if(NOT "$ENV{CMAKE_TOOLCHAIN_FILE}" MATCHES "vcpkg")
            set(HELLOIMGUI_FREETYPE_STATIC ON CACHE BOOL "" FORCE)
        endif()
    endif()
endfunction()

# Check if freetype is available: ON by default, except on Android and MinGW
# On MacOS, if building a distributable wheel with cibuilwheel for a version less than 14.0, we disable freetype
function(ibd_check_freetype_availability result_var)
    # freetype is available by default
    set(${result_var} ON PARENT_SCOPE)

    # Except on Android and MinGW
    # Freetype is not available on Android. The mix SDL + Freetype cause issues (cannot find SDL.h)
    # Freetype currently fails to build on MinGW
    if(ANDROID OR MINGW)
        set(${result_var} OFF PARENT_SCOPE)
        return()
    endif()

hello_imgui_build_lib.cmake: build a static version of freetype (without Brolti) and links with it

hello_imgui_build_lib.cmake will

https://github.com/pthom/hello_imgui/blob/6f96155f5cb90dff93da21444d8586aca1ade17b/hello_imgui_cmake/hello_imgui_build_lib.cmake#L391-L421

        if(HELLOIMGUI_FREETYPE_STATIC AND NOT HELLOIMGUI_FETCH_FORBIDDEN)
            message("Forcing download of Freetype because HELLOIMGUI_FREETYPE_STATIC is ON")
            set(download_freetype ON)
        endif()

        if (download_freetype)
            message(STATUS "HelloImGui: downloading and building freetype")

            set(backup_shared_lib ${BUILD_SHARED_LIBS})
            set(BUILD_SHARED_LIBS OFF CACHE BOOL "" FORCE)

            if (NOT HELLOIMGUI_INSTALL)
                set(SKIP_INSTALL_ALL ON CACHE INTERNAL "" FORCE) # disable Freetype install
            endif()

            include(FetchContent)
            if(IOS)
                set(FT_DISABLE_HARFBUZZ ON CACHE BOOL "" FORCE)
                set(FT_DISABLE_BROTLI ON CACHE BOOL "" FORCE)
            endif()
            FetchContent_Declare(
                freetype
                GIT_REPOSITORY https://github.com/freetype/freetype.git
                GIT_TAG        VER-2-13-2
                GIT_PROGRESS TRUE
            )
            FetchContent_MakeAvailable(freetype)
            set(freetype_linked_library freetype)

and then freetype_linked_library is used with:

    target_link_libraries(imgui PUBLIC ${freetype_linked_library})

Wheel build log:

Below, I include extracts from the build logs on GitHub CI

We can see this lines, which shows that

  1. Freetype was built
  2. However, the link did not use our built version, but instead a version at '/opt/homebrew/Cellar/brotli/1.1.0/lib/libbrotlidec.1.1.0.dylib which is not for the correct architecture

Some extracts:

Build of freetype: https://github.com/pthom/imgui_bundle/actions/runs/11166142777/job/31039470761#step:3:5460 " -- HelloImGui: downloading and building freetype" https://github.com/pthom/imgui_bundle/actions/runs/11166142777/job/31039470761#step:3:7678 " [173/308] Building C object _deps/freetype-build/CMakeFiles/freetype.dir/src/autofit/autofit.c.o"

https://github.com/pthom/imgui_bundle/actions/runs/11166142777/job/31039470761#step:3:7868

Link step, extract

    [306/308] Linking CXX shared module _imgui_bundle.cpython-312-darwin.so
    ld: warning: ignoring duplicate libraries: '-ldl', 'external/hello_imgui/hello_imgui/src/hello_imgui/libimgui_test_engine.a', 'external/immapp/immapp/libimmapp.a', 'external/libimgui.a', 'external/libimgui_md.a', 'external/libimgui_node_editor.a'
    ld: warning: ignoring file '/opt/homebrew/Cellar/libpng/1.6.44/lib/libpng16.16.dylib': found architecture 'arm64', required architecture 'x86_64'
    ld: warning: ignoring file '/opt/homebrew/Cellar/libpng/1.6.44/lib/libpng16.16.dylib': found architecture 'arm64', required architecture 'x86_64'
    ld: warning: ignoring file '/opt/homebrew/Cellar/harfbuzz/9.0.0/lib/libharfbuzz.0.dylib': found architecture 'arm64', required architecture 'x86_64'
    ld: warning: ignoring file '/opt/homebrew/Cellar/brotli/1.1.0/lib/libbrotlidec.1.1.0.dylib': found architecture 'arm64', required architecture 'x86_64'
    [307/308] cd /var/folders/0g/hj_q_pzx65bbjnslxz9n0src0000gn/T/tmplel4blzs/build && /opt/homebrew/Cellar/cmake/3.30.3/bin/cmake -E copy /var/folders/0g/hj_q_pzx65bbjnslxz9n0src0000gn/T/tmplel4blzs/build/_imgui_bundle.cpython-312-darwin.so /Users/runner/work/imgui_bundle/imgui_bundle/bindings/imgui_bundle/_imgui_bundle.cpython-312-darwin.so
    [308/308] cd /var/folders/0g/hj_q_pzx65bbjnslxz9n0src0000gn/T/tmplel4blzs/build && /opt/homebrew/Cellar/cmake/3.30.3/bin/cmake -E copy /var/folders/0g/hj_q_pzx65bbjnslxz9n0src0000gn/T/tmplel4blzs/build/_imgui_bundle.cpython-312-darwin.so /private/var/folders/0g/hj_q_pzx65bbjnslxz9n0src0000gn/T/cibw-run-k97iif_g/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/imgui_bundle/_imgui_bundle.cpython-312-darwin.so

Those warnings explain the failure:

    ld: warning: ignoring file '/opt/homebrew/Cellar/libpng/1.6.44/lib/libpng16.16.dylib': found architecture 'arm64', required architecture 'x86_64

Conclusion

We did build a static version of freetype. Unfortunately, the lines

     set(freetype_linked_library freetype)
    target_link_libraries(imgui PUBLIC ${freetype_linked_library})

Do not give precedence to our static libraries and instead it uses a library found on the OS which does not have the good architecture.

pthom commented 1 month ago

I'm trying two patches:

For more details, see https://chatgpt.com/share/66ffb718-6408-8004-be5b-9e74064a8709

Please keep in touch and answer the two questions below:

  1. Try the previous wheel, after having installed freetype 
    brew install freetype

The idea is that I guess that the stock version of freetype is built with harfbuzz, brotli and png. Maybe they will be found once you run. Keep me informed whether it works or not.

  1. Try the wheels at https://github.com/pthom/imgui_bundle/actions/runs/11178781726/job/31077057633 They are using those two patches and I hope they will work.

Conclusions and follow-up

Effect of disabling features in freetype, according to ChatGPT:

Disabling Harfbuzz, Brotli, and PNG in FreeType primarily affects fonts with advanced layout features and bitmap compression.

Harfbuzz: Without it, complex text shaping (for scripts like Arabic, Hindi, etc.) won’t work, but most Latin-based fonts will be unaffected. Brotli: Disabling it affects fonts using Brotli compression, mostly impacting modern Google Fonts or web-based fonts. PNG: Disabling PNG impacts bitmap glyphs that use PNG, typically affecting embedded bitmaps in some fonts (e.g., emoji fonts).

For most general-purpose fonts, the impact will be minimal.

dwgrth commented 1 month ago

Hi! I also happened to encounter this problem today.

I can verify that using the wheel at: https://github.com/pthom/imgui_bundle/actions/runs/11178781726/artifacts/2015448630 imgui_bundle-1.5.2-cp312-cp312-macosx_14_0_x86_64.whl

Fixed the problem for me. Thank you!

pthom commented 1 month ago

@dwgrth : Thanks! If you have time, could you also try other the alternative (brew install freetype + install the standard wheel), and tell me if it works for you.

Thanks!

virtuald commented 1 month ago

Maybe you could uninstall the system freetype? I hate cmake.

virtuald commented 1 month ago

However, the wheel fixed the problem for me too.

virtuald commented 1 month ago

Are you planning to do a release with this fix in the near future?

pthom commented 1 month ago

Yes, I think I will make a release in about a month.

virtuald commented 1 month ago

So.. no.