Windows: MSVC linker error LNK2001: unresolved external symbol "void * _tsrm_ls_cache" - when building PHP extension

sjayexec commented 1 day ago

Description

Building our PHP extension with MSVC (VS Build Tools 2019 + Windows 10 SDK) fails with the linker unable to find _tsrm_ls_cache symbol. We are linking against the php8ts.lib in our cmake configuration.

The PHP runtime is built with ZTS enabled and our extension is also being built with the same compilation flags (ZTS=1). We have this flag enabled: ZEND_ENABLE_STATIC_TSRMLS_CACHE=1 which seems to be causing this problem. Removing this TSRM flag makes these linker errors go away. This flag works fine when we compile our extension on Linux with Clang.

MSVC linker errors:

[build] Starting build
[proc] Executing command: "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" --build c:/path/build --config Release --target ALL_BUILD -j 22 --
[build] Microsoft (R) Build Engine version 16.11.2+f32259642 for .NET Framework
[build] Copyright (C) Microsoft Corporation. All rights reserved.

[build] C:\work\PHP_ENV\php-sdk\php-src-build-dir\vs16\x64\php-src\Zend\zend_gc.h(155,22): warning C4244: '=': conversion from '__int64' to 'int', possible loss of data [C:\workspace\abc.vcxproj]
[build]      Creating library C:/abc.lib and object C:/abc.exp
[build] file1.cpp.obj : error LNK2001: unresolved external symbol "void * _tsrm_ls_cache" (?_tsrm_ls_cache@@3PEAXEA) [C:\workspace\abc.vcxproj]
[build] file2.obj : error LNK2001: unresolved external symbol "void * _tsrm_ls_cache" (?_tsrm_ls_cache@@3PEAXEA) [C:\workspace\abc.vcxproj]
[build] file3.obj : error LNK2001: unresolved external symbol "void * _tsrm_ls_cache" (?_tsrm_ls_cache@@3PEAXEA) [C:\workspace\abc.vcxproj]
[build] file4.obj : error LNK2001: unresolved external symbol "void * _tsrm_ls_cache" (?_tsrm_ls_cache@@3PEAXEA) [C:\workspace\abc.vcxproj]
[build] file5.obj : error LNK2001: unresolved external symbol "void * _tsrm_ls_cache" (?_tsrm_ls_cache@@3PEAXEA) [C:\workspace\abc.vcxproj]
[build]     Hint on symbols that are defined and could potentially match:
[build]       _tsrm_ls_cache
[build] C:\workspace\Release\abc.dll : fatal error LNK1120: 1 unresolved externals [C:\workspace\abc.vcxproj]
[proc] The command: "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" --build c:/path/build --config Release --target ALL_BUILD -j 22 -- exited with code: 1
[driver] Build completed: 00:00:30.345
[build] Build finished with exit code 1

PHP SDK & MSVC version

PHP SDK 2.3.0
ARCH x64
Visual C++:         14.29.30156.0

PHP version:

PS > php -v
PHP 8.3.12-dev (cli) (built: Dec  2 2024 17:43:09) (ZTS Visual C++ 2019 x64)
Copyright (c) The PHP Group
Zend Engine v4.3.12-dev, Copyright (c) Zend Technologies

configure command used to build PHP runtime:

configure --disable-phpdbg --enable-intl --enable-mbstring --enable-opcache --enable-pdo --enable-soap --enable-sockets --with-curl --with-mysqli --with-openssl --with-pdo-mysql --with-pdo-sqlite --with-sqlite3 --enable-fileinfo --enable-embed --enable-zts

We are using these flags on Windows when building our extension:

_CRT_SECURE_NO_WARNINGS
NOMINMAX
PHP_WIN32=1
ZEND_WIN32=1
ZTS=1   # ZTS may not always be on, but main\config.w32.h doesn't seem to carry it unlike main/php_config.h on Linux
ZEND_ENABLE_STATIC_TSRMLS_CACHE=1

I looked into the symbols inside php8ts.lib library file using dumpbin /SYMBOLS php8ts.lib command, there is no _tsrm_ls_cache symbol in it. But I do find a symbol for it in php8embed.lib library. So, I tried disabling embed extension and rebuild PHP but the php8ts.lib still does not seem to have _tsrm_ls_cache.

I'm not sure what's going wrong here. We are not doing anything special w.r.t TSRM. We have ZEND_TSRMLS_CACHE_DEFINE() if ZTS is enabled in our main extension source code and then in PHP_GINIT_FUNCTION of our extension, we do ZEND_TSRMLS_CACHE_UPDATE(). This works fine on Linux.

To rule out any misconfiguration of my env, I tried this multiple times - with Debug/Release variants, but I see the same result. Could you please help us understand this issue with linking against PHP libraries. Thanks in advance!

PHP Version

PHP 8.3.12-dev

Operating System

Windows 10 SDK (10.0.22631.4460)

Note: Edited the description to keep all observations and printouts from release build to avoid confusion. It happens both for debug & release builds.

cmb69 commented 1 day ago

You may also need to declare the symbol, like

https://github.com/php/php-src/blob/51e0920e4513c6b6fa0d0b8141d5703b3db3abbe/ext/skeleton/php_skeleton.h#L11-L13

By the way, do you have any particular reason for using CMake for your extension, instead of the customary config.m4/config.w32 autotools(-like) builds?

sjayexec commented 22 hours ago

We do use the ZEND_TSRMLS_CACHE_EXTERN() macro in the php_<extension>.h header file of our extension to declare it there while it is defined in the <extension>.c file. All of this works already on Linux.

It is a proprietary extension, so maybe that was the reason to build it outside the PHP build system, I would guess. On Linux, we link only against the PHP headers and let the symbols resolve at runtime when the extension is loaded. But on Windows, this is not possible it seems. So in our CMake configuration, we are linking against the PHP libraries directly and see this error.

We also have a couple more unresolved symbol errors at link-time, I will share more details shortly.

strcasestr(): I know fileinfo extension provides some functions for Windows target that are not available natively. But even after enabling fileinfo extension during PHP build and including the extension's file.h header file (which declares the function) in our code, the linker fails to find this. I added 'php_fileinfo.lib' as a link library when linking our extension, but no success.
zend_compile_file: The linker fails to resolve this symbol too on Windows. Not sure if any extra headers need to be included or what compared to linux as it works on that platform.

sjayexec commented 19 hours ago

Update: Ignore this observation - see next comment You can see here that when we use zend_compile_file stuff from Zend/zend_compile.h, the linker fails to link it:

[build] profiler.obj : error LNK2001: unresolved external symbol "void * _tsrm_ls_cache" (?_tsrm_ls_cache@@3PEAXEA) [C:\workspace\abc.vcxproj]
[build]     Hint on symbols that are defined and could potentially match:
[build]       _tsrm_ls_cache
[build] instana.obj : error LNK2019: unresolved external symbol __imp__zend_compile_file referenced in function zm_startup_instana [C:\workspace\abc.vcxproj]
[build] C:\workspace\abc.dll : fatal error LNK1120: 2 unresolved externals [C:\workspace\abc.vcxproj]

Upon closer look, the linker seems to look for __imp__zend_compile_file symbol while the php8ts.lib has __imp_zend_compile_file symbol - extra underscore (_) before imp and zend in the symbol name the linker is searching for.

sjayexec commented 17 hours ago

Please ignore zend_compile_file related error, we were wrongly declaring _zend_compile_file() in our extension with prefix underscore. But this is coming from a working linux code, I have to understand why _zend_compile_file() declaration & usage works on linux but on windows, we need to use only zend_compile_file() - something to do with the compilers perhaps.

So apart from TSRM symbol error, another issue is with strcasestr() function symbol not being located. Could you please tell me what is the right way to build PHP with fileinfo extension and then use strcasestr() in our extension's code?

We built PHP with fileinfo extension and in our code, we were including this header file like this:

#ifdef _WIN32
#include <ext/fileinfo/libmagic/file.h>
#endif

and try to link our extension with php_fileinfo.lib But linker fails to find the strcasestr() symbol.

cmb69 commented 14 hours ago

Please ignore zend_compile_file related error

That may have basically the same reason as tsrm_ls_cache. Both are variables, not functions, so they need __declspec(dllexport) when building the DLL (or you need to add .def files). I guess that you are not properly including the required headers, or don't have the appriate macros defined. See e.g. https://heap.space/xref/php-src/Zend/zend_config.w32.h?r=79e4ca1e#47.

strcasestr() is defined in strcasestr.c; maybe you don't link with this file.

Anyhow, this doesn't look like a bug report (nor a feature request), but rather a support question. These should be asked on the available support channels, in this case possibly the Windows Internals List.

sjayexec commented 13 hours ago

The zend_compile_file error is something wrong on our side. We had _declspec(dllimport) _zend_compile_file - but the actual name in Zend is zend_compile_file. We were not trying to forward declare but create a local variable, so using _declspec(dllimport) is wrong.

But for _tsrm_ls_cache, we are nowhere using this variable explicitly; we use the TSRM DEFINE, EXTERN, UPDATE macros the usual way most extensions are doing. We enabled PHP_WIN32, ZEND_WIN32, ZEND_ENABLE_STATIC_TSRMLS_CACHE macros and we check for ZTS before doing #include TSRM.h.

I will contact the Windows mailing list, thanks for your support!

cmb69 commented 12 hours ago

I've totally overlooked this so far:

[build] file5.obj : error LNK2001: unresolved external symbol "void * _tsrm_ls_cache" (?_tsrm_ls_cache@@3PEAXEA) [C:\workspace\abc.vcxproj]
[build]     Hint on symbols that are defined and could potentially match:
[build]       _tsrm_ls_cache

There is obviously a name mangling problem.

$ undname ?_tsrm_ls_cache@@3PEAXEA
Microsoft (R) C++ Name Undecorator
Copyright (C) Microsoft Corporation. All rights reserved.

Undecoration of :- "?_tsrm_ls_cache@@3PEAXEA"
is :- "void * __ptr64 __ptr64 _tsrm_ls_cache"

Somewhere the compiler doesn't pick up the proper declaration (by (ZEND_)TSRMLS_CACHE_DEFINE()), and mangles the name.

php / php-src