Invalid cache assertion building freetype port

[HCLJason]

I'm trying out an updated freetype port using cmake. When I run embuilder build freetype, I get a cache lock error on the first assert (line 39 in cache.py).

I added print(dict(os.environ)), right before it, and EM_CACHE_IS_LOCKED did not show up in it.

After commenting out the assert, it seems to hang when building freetype.

Please include the following in your bug report:

Version of emscripten/emsdk: Please include the output emcc -v here

3.1.23

Failing command line in full: If this is compile or link-time failure please include the full failing command along with its entire output.

python3 ./embuilder.py build zlib freetype

embuilder:INFO: building freetype ports:INFO: retrieving port: freetype from https://download.savannah.gnu.org/releases/freetype/freetype-2.12.1.tar.gz ports:INFO: unpacking port: freetype cache:INFO: generating port: sysroot/lib/wasm32-emscripten/libfreetype.a... (this will be cached in "/local/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libfreetype.a" for subsequent builds) configure: cmake -B/local/emsdk/upstream/emscripten/cache/ports-builds/freetype -S/local/emsdk/upstream/emscripten/cache/ports/freetype/freetype-2.12.1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/local/emsdk/upstream/emscripten/cache/ports-builds/freetype -DCMAKE_CXX_FLAGS="-I/local/emsdk/upstream/emscripten/system/lib/libc/musl/include/" -DCMAKE_C_FLAGS="-I/local/emsdk/upstream/emscripten/system/lib/libc/musl/include/" -DCMAKE_TOOLCHAIN_FILE=/local/emsdk/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake -DCMAKE_CROSSCOMPILING_EMULATOR=/local/emsdk/node/14.18.2_64bit/bin/node;--experimental-wasm-threads -- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE) CMake Warning at builds/cmake/FindHarfBuzz.cmake:99 (message): Required version (2.0.0) is higher than found version () Call Stack (most recent call first): CMakeLists.txt:253 (find_package)

-- Found ZLIB: /local/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libz.a (found version "1.2.12") -- Could NOT find PNG (missing: PNG_LIBRARY PNG_PNG_INCLUDE_DIR) -- Could NOT find BZip2 (missing: BZIP2_LIBRARIES BZIP2_INCLUDE_DIR) -- Could NOT find BrotliDec (missing: BROTLIDEC_INCLUDE_DIRS BROTLIDEC_LIBRARIES) -- Looking for unistd.h -- Looking for unistd.h - not found

Traceback (most recent call last): File "/local/emsdk/upstream/emscripten/emcc.py", line 4145, in sys.exit(main(sys.argv)) File "/usr/lib/python3.8/contextlib.py", line 75, in inner return func(*args, kwds) File "/local/emsdk/upstream/emscripten/emcc.py", line 4138, in main ret = run(args) File "/local/emsdk/upstream/emscripten/emcc.py", line 1192, in run phase_calculate_system_libraries(state, linker_arguments, linker_inputs, newargs) File "/usr/lib/python3.8/contextlib.py", line 75, in inner return func(*args, *kwds) File "/local/emsdk/upstream/emscripten/emcc.py", line 2860, in phase_calculate_system_libraries extra_files_to_link += system_libs.calculate(all_linker_inputs, newargs, forced=state.forced_stdlibs) File "/local/emscripten/tools/system_libs.py", line 2132, in calculate libs_to_link = get_libs_to_link(args, forced, only_forced) File "/local/emscripten/tools/system_libs.py", line 2068, in get_libs_to_link add_library('libGL') File "/local/emscripten/tools/system_libs.py", line 2014, in add_library libs_to_link.append((lib.get_link_flag(), need_whole_archive)) File "/local/emscripten/tools/system_libs.py", line 379, in get_link_flag fullpath = self.build() File "/local/emscripten/tools/system_libs.py", line 371, in build return shared.Cache.get(self.get_path(), self.do_build, force=USE_NINJA == 2, quiet=USE_NINJA) File "/local/emscripten/tools/cache.py", line 145, in get with self.lock(shortname): File "/usr/lib/python3.8/contextlib.py", line 113, in enter return next(self.gen) File "/local/emscripten/tools/cache.py", line 65, in lock self.acquire_cache_lock(reason) File "/local/emscripten/tools/cache.py", line 42, in acquire_cache_lock assert 'EM_CACHE_IS_LOCKED' not in os.environ, f'attempt to lock the cache while a parent process is holding the lock ({reason})' AssertionError: attempt to lock the cache while a parent process is holding the lock (sysroot/lib/wasm32-emscripten/libGL.a) make[1]: [CMakeFiles/cmTC_d8893.dir/build.make:88: cmTC_d8893.js] Error 1 make[1]: Leaving directory '/local/emscripten/cache/ports-builds/freetype/CMakeFiles/CMakeTmp' make: *** [Makefile:121: cmTC_d8893/fast] Error 2

[sbc100]

We try to avoid using any external tools when building ports, since that would mean that any users of the port then need to have that tool installed. So we don't use make or ninja or cmake in tools/ports anywhere, since many emsdk users don't have any of those tools installed.

Is there some reason why the freetype is hard to maintain in its current form?

[HCLJason]

The port is version 2.6.1, which doesn't have a bunch of security fixes.

https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=freetype

I see a ninja-using PR into tools/ports here: https://github.com/emscripten-core/emscripten/pull/17809

[sbc100]

Yes we are considering adding ninja as a dependency, but have not actually taken that step.

Note that with that PR the existing ports build system will generate ninja directly (we still don't plan on shipping cmake).

Is there some issue with upgrading freetype using the current (internal) build system for ports?

[sbc100]

If you goal is to just to upgrade freetype I think we can probably just do that without adding support for external built tools.

We can also discuses the benefits of using an external build tool separately.

[HCLJason]

I tried that. The initial issue is duplicate zlib symbols, so naturally I swapped the compile to use the system zlib.

I tried both using it as a compiler flag ('-DFT_REQUIRE_ZLIB') and adding it to the ftoption.h file: with open(os.path.join(source_path, 'include', 'freetype', 'config','ftoption.h'), 'a') as ftheader: ftheader.write('#define FT_CONFIG_OPTION_SYSTEM_ZLIB')

At this point, I don't think this needs to get fixed, but I'll try to get an updated zlib + freetype port for you soon.

[sbc100]

Thanks, if need help with the update please reach out. Perhaps it might be worth opening a bug to update freetype? (and we can close this one)

[HCLJason]

I'll open up an issue once I have the ports ready to go.

For the freetype update, there's 3 ftsystem files - unix, win32, and base. I assume that I want base, right?

Where did you get the ftconf.h file? It looks like a mashup of ftoption.h from emconfigure configure and something else.

[sbc100]

I imagine you will want base + unix

[HCLJason]

Using unix has issues, but base works fine. I have PRs in for both zlib and freetype updates.

https://github.com/emscripten-core/emscripten/pull/18087 https://github.com/emscripten-core/emscripten/pull/18088

[Oipo]

I think I'm running into a related issue with the standard ports SDL_ttf/SDL_mixer, so I'll attach my log here.

emscripten cache issue.txt