Closed pananton closed 2 years ago
pananton
commented
2 years ago Forgot to mention, that if I directly login to runner's machine with the same user ('gitlab' in my case) and execute conan create in a powershell then everything works just fine. The problem is only when recipe is built as part of CI, so I'm almost sure that the problem is in it's settings, but maybe someone had the same issue..
pananton
commented
2 years ago I've changed libiconv recipe to use msys2/20210105 as dependency and everything works. Then I've tried to use msys2/20210105 recipe but make it download the same archive as current msys2/cci.latest (through conandata.yml) and received the same error. So I belive that the problem relates to msys2 itself. Any advices/opinions are appreciated)
SSE4
commented
2 years ago is it possible that it spawns a dialog box (like runtime error, assertion failed or DLL load error)?
pananton
commented
2 years ago If I directly login to the CI runner machine with the same user as used by the runner and perform conan create directly no dialog boxes are opened... I've tried even to perform command that stucks directly and it also worked just fine. I mean, this command:
C:\.conan\8c5bea\1\bin\msys64\usr\bin\bash.exe --login -c ^"cd \^"/c/users/gitlab/.conan/data/libiconv/1.16/_/_/build/3fb49604f9c2f729b85ba3115852006824e72cab/source_subfolder\^" ^&^& PATH=\^"/c/.conan/8c5bea/1/bin/msys64/usr/bin:$PATH\^" ^&^& /c/users/gitlab/.conan/data/libiconv/1.16/_/_/build/3fb49604f9c2f729b85ba3115852006824e72cab/source_subfolder/configure --enable-static --disable-shared --prefix=C:/Users/gitlab/.conan/data/libiconv/1.16/_/_/package/3fb49604f9c2f729b85ba3115852006824e72cab --host=x86_64-w64-mingw32 ^"Also I've tried to set custom CONAN_USER_HOME / CONAN_USER_HOME_SHORT , because runner is launched as a windows service and may not have a home directory or may be used concurrently by other jobs - and it also did not work (
pananton
commented
2 years ago Additional information. As I mentioned earlier build stucks during configure step (when configure script is working). I've looked through this script and find out that it creates conftest.c file and tries to compile it to check C compiler features. When build stucks (note that cl.exe exists in the active processes) conftest.c looks like this:
/* confdefs.h */
#define PACKAGE_NAME "libiconv"
#define PACKAGE_TARNAME "libiconv"
#define PACKAGE_VERSION "1.16"
#define PACKAGE_STRING "libiconv 1.16"
#define PACKAGE_BUGREPORT ""
#define PACKAGE_URL ""
#define PACKAGE "libiconv"
#define VERSION "1.16"
#define STDC_HEADERS 1
#define HAVE_SYS_TYPES_H 1
#define HAVE_SYS_STAT_H 1
#define HAVE_STDLIB_H 1
#define HAVE_STRING_H 1
#define HAVE_MEMORY_H 1
#define HAVE_INTTYPES_H 1
#define HAVE_STDINT_H 1
#define __EXTENSIONS__ 1
#define _ALL_SOURCE 1
#define _DARWIN_C_SOURCE 1
#define _GNU_SOURCE 1
#define _NETBSD_SOURCE 1
#define _OPENBSD_SOURCE 1
#define _POSIX_PTHREAD_SEMANTICS 1
#define __STDC_WANT_IEC_60559_ATTRIBS_EXT__ 1
#define __STDC_WANT_IEC_60559_BFP_EXT__ 1
#define __STDC_WANT_IEC_60559_DFP_EXT__ 1
#define __STDC_WANT_IEC_60559_FUNCS_EXT__ 1
#define __STDC_WANT_IEC_60559_TYPES_EXT__ 1
#define __STDC_WANT_LIB_EXT2__ 1
#define __STDC_WANT_MATH_SPEC_FUNCS__ 1
#define _TANDEM_SOURCE 1
#define _HPUX_ALT_XOPEN_SOCKET_API 1
#define _DARWIN_USE_64_BIT_INODE 1
#define LT_OBJDIR ".libs/"
#define INSTALLPREFIX "C:/Users/gitlab/conan-recipe/.conan/data/libiconv/1.16/_/_/package/3fb49604f9c2f729b85ba3115852006824e72cab"
#define HAVE_VISIBILITY 0
#define ICONV_CONST
#define HAVE_MBSTATE_T 1
#define HAVE_WCHAR_T 1
#define HAVE_MBRTOWC 1
#define HAVE_WCRTOMB 1
#define HAVE_SETLOCALE 1
#define HAVE_STDLIB_H 1
#define HAVE_STRING_H 1
#define HAVE_LIMITS_H 1
#define HAVE_SYS_STAT_H 1
#define HAVE_WCHAR_H 1
#define HAVE_STDINT_H 1
#define HAVE_GETCWD 1
#define HAVE__SET_INVALID_PARAMETER_HANDLER 1
#define HAVE_WORKING_O_NOATIME 0
#define HAVE_WORKING_O_NOFOLLOW 0
#define HAVE_WCHAR_H 1
#define WORDS_LITTLEENDIAN 1
#define HAVE_ALLOCA 1
#define DOUBLE_SLASH_IS_DISTINCT_ROOT 1
#define HAVE_ENVIRON_DECL 1
#define HAVE_DECL_STRERROR_R 0
#define pid_t int
#define mode_t int
#define HAVE_STDLIB_H 1
#define MALLOC_0_IS_NONNULL 1
#define HAVE_UNSIGNED_LONG_LONG_INT 1
#define HAVE_LONG_LONG_INT 1
#define HAVE_MSVC_INVALID_PARAMETER_HANDLER 1
#define HAVE_DECL_SETENV 0
#define ssize_t int
#define uid_t int
#define gid_t int
#define USER_LABEL_PREFIX
#define nlink_t int
#define HAVE__BOOL 1
#define HAVE_WINT_T 1
#define restrict /**/
/* end confdefs.h. */
#include <string.h>This is the example how configure script tries to determine location of the <string.h> header. If i kill cl.exe the following line appears in my CI job log:
checking absolute name of <string.h>... "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\ucrt\\string.h"The build process continues, but later stucks again. The content of conftest.c now is:
/* confdefs.h */
#define PACKAGE_NAME "libiconv"
#define PACKAGE_TARNAME "libiconv"
#define PACKAGE_VERSION "1.16"
#define PACKAGE_STRING "libiconv 1.16"
#define PACKAGE_BUGREPORT ""
#define PACKAGE_URL ""
#define PACKAGE "libiconv"
#define VERSION "1.16"
#define STDC_HEADERS 1
#define HAVE_SYS_TYPES_H 1
#define HAVE_SYS_STAT_H 1
#define HAVE_STDLIB_H 1
#define HAVE_STRING_H 1
#define HAVE_MEMORY_H 1
#define HAVE_INTTYPES_H 1
#define HAVE_STDINT_H 1
#define __EXTENSIONS__ 1
#define _ALL_SOURCE 1
#define _DARWIN_C_SOURCE 1
#define _GNU_SOURCE 1
#define _NETBSD_SOURCE 1
#define _OPENBSD_SOURCE 1
#define _POSIX_PTHREAD_SEMANTICS 1
#define __STDC_WANT_IEC_60559_ATTRIBS_EXT__ 1
#define __STDC_WANT_IEC_60559_BFP_EXT__ 1
#define __STDC_WANT_IEC_60559_DFP_EXT__ 1
#define __STDC_WANT_IEC_60559_FUNCS_EXT__ 1
#define __STDC_WANT_IEC_60559_TYPES_EXT__ 1
#define __STDC_WANT_LIB_EXT2__ 1
#define __STDC_WANT_MATH_SPEC_FUNCS__ 1
#define _TANDEM_SOURCE 1
#define _HPUX_ALT_XOPEN_SOCKET_API 1
#define _DARWIN_USE_64_BIT_INODE 1
#define LT_OBJDIR ".libs/"
#define INSTALLPREFIX "C:/Users/gitlab/conan-recipe/.conan/data/libiconv/1.16/_/_/package/3fb49604f9c2f729b85ba3115852006824e72cab"
#define HAVE_VISIBILITY 0
#define ICONV_CONST
#define HAVE_MBSTATE_T 1
#define HAVE_WCHAR_T 1
#define HAVE_MBRTOWC 1
#define HAVE_WCRTOMB 1
#define HAVE_SETLOCALE 1
#define HAVE_STDLIB_H 1
#define HAVE_STRING_H 1
#define HAVE_LIMITS_H 1
#define HAVE_SYS_STAT_H 1
#define HAVE_WCHAR_H 1
#define HAVE_STDINT_H 1
#define HAVE_GETCWD 1
#define HAVE__SET_INVALID_PARAMETER_HANDLER 1
#define HAVE_WORKING_O_NOATIME 0
#define HAVE_WORKING_O_NOFOLLOW 0
#define HAVE_WCHAR_H 1
#define WORDS_LITTLEENDIAN 1
#define HAVE_ALLOCA 1
#define DOUBLE_SLASH_IS_DISTINCT_ROOT 1
#define HAVE_ENVIRON_DECL 1
#define HAVE_DECL_STRERROR_R 0
#define pid_t int
#define mode_t int
#define HAVE_STDLIB_H 1
#define MALLOC_0_IS_NONNULL 1
#define HAVE_UNSIGNED_LONG_LONG_INT 1
#define HAVE_LONG_LONG_INT 1
#define HAVE_MSVC_INVALID_PARAMETER_HANDLER 1
#define HAVE_DECL_SETENV 0
#define ssize_t int
#define uid_t int
#define gid_t int
#define USER_LABEL_PREFIX
#define nlink_t int
#define HAVE__BOOL 1
#define HAVE_WINT_T 1
#define restrict /**/
#define _USE_STD_STAT 1
#define HAVE_DECL_CLEARERR_UNLOCKED 0
#define HAVE_DECL_FEOF_UNLOCKED 0
#define HAVE_DECL_FERROR_UNLOCKED 0
#define HAVE_DECL_FFLUSH_UNLOCKED 0
#define HAVE_DECL_FGETS_UNLOCKED 0
#define HAVE_DECL_FPUTC_UNLOCKED 0
#define HAVE_DECL_FPUTS_UNLOCKED 0
#define HAVE_DECL_FREAD_UNLOCKED 0
#define HAVE_DECL_FWRITE_UNLOCKED 0
#define HAVE_DECL_GETC_UNLOCKED 0
#define HAVE_DECL_GETCHAR_UNLOCKED 0
#define HAVE_DECL_PUTC_UNLOCKED 0
#define HAVE_DECL_PUTCHAR_UNLOCKED 0
#define GNULIB_CANONICALIZE_LGPL 1
#define GNULIB_TEST_CANONICALIZE_FILE_NAME 1
#define GNULIB_TEST_REALPATH 1
#define DOUBLE_SLASH_IS_DISTINCT_ROOT 1
#define GNULIB_TEST_ENVIRON 1
#define HAVE_DECL_PROGRAM_INVOCATION_NAME 0
#define HAVE_DECL_PROGRAM_INVOCATION_SHORT_NAME 0
#define HAVE_DECL___ARGV 1
#define GNULIB_TEST_LSTAT 1
#define GNULIB_TEST_MALLOC_POSIX 1
#define HAVE_MBSTATE_T 1
#define GNULIB_MSVC_NOTHROW 1
#define HAVE_DECL_PROGRAM_INVOCATION_NAME 0
#define HAVE_DECL_PROGRAM_INVOCATION_SHORT_NAME 0
#define HAVE_RAISE 1
#define GNULIB_TEST_RAISE 1
#define GNULIB_TEST_READ 1
#define GNULIB_TEST_READLINK 1
#define HAVE_SEARCH_H 1
#define GNULIB_SIGPIPE 1
#define GNULIB_TEST_SIGPROCMASK 1
#define ssize_t int
#define GNULIB_TEST_STAT 1
#define __USE_MINGW_ANSI_STDIO 1
/* end confdefs.h. */
#include <stdio.h>If cl.exe is killed again the build process continues and the next line is CI job log is:
checking absolute name of <stdio.h>... "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\ucrt\\stdio.h"Then process stucks again, killing cl.exe works, etc until build is finished. In all cases the content of conftest.c is very similar/ Among several defines there is only one line including some header (no main function etc). Other checks (when conftest.c contains main function) successfully pass.
pananton
commented
2 years ago Looks like the process stucks on the following command from configure script created by autoconf (I've replaced bash variables with actual values here):
gl_dirsep_regex='[/\\]'
gl_make_literal_regex_sed='s,[]$^\\.*/[],\\&,g'
gl_header_literal_regex=`echo 'stdio.h' | sed -e "$gl_make_literal_regex_sed"`
gl_absolute_header_sed="/${gl_dirsep_regex}${gl_header_literal_regex}/"'{
s/.*"\(.*'"${gl_dirsep_regex}${gl_header_literal_regex}"'\)".*/\1/
s|^/[^/]|//&|
p
q
}'
gl_cv_absolute_stdio_h=`(eval "cl -E conftest.c") 2>config.log | sed -n "$gl_absolute_header_sed"`As you can see from the last line, configure calls cl.exe as C preprocessor to parse conftest.c (remember, that it contains include <stdio.h>). Then at some moment process stucks until cl.exe is manually killed. Because absolute path to stdio.h is correctly defined (this can be seen in conan's output too), I think that the problem is in sed command (I don't know it well, so can't say why it can stuck).. Any ideas?) Btw, cl.exe version is Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30141 for x64
SSE4
commented
2 years ago if you execute just cl -E conftest.c, does it complete?
pananton
commented
2 years ago Yes
pananton
commented
2 years ago @SSE4 Hello! I finally figured out exact source of problem. As I've mentioned, libiconv/1.16 builds just fine with msys2/20210105 (I simply modified original recipe from conan-center-index to use it), but stucks when building it with msys2/cci.latest. Then I installed both msys2 versions in my runner's local cache and start to copy files from msys2/20210105 usr/bin to the same directory of msys2/cci.latest. Using something like binary search algorithm when copying, I've discovered, that replacing single file msys-2.0.dll is enough to fix the problem. I mean, if I install msys2/cci.latest,replace it's usr/bin/msys-2.0.dll to the same file from msys2/20210105 then libiconv is built as expected. What should I do next?) Maybe I should post this to conan-center-index?
SSE4
commented
2 years ago I think it clearly indicates a bug in msys itself, consider opening an issue against MSYS2: https://github.com/msys2/msys2-packages/issues
pananton
commented
2 years ago Created an issue against msys2.. https://github.com/msys2/MSYS2-packages/issues/2893 . They already have a similar one, hope the problem will be solved soon. Thanks for advices.
Hello! I have a problem with building libiconv and some other libraries which use msys2 on windows as part of gitlab CI job (run in powershell). Probably the cause of error is specific to gitlab, but maybe you can give some advice where to look for solution. When my CI job executes
conan create libiconv/1.16@ .on windows I see the following output:After the last line the building stucks forever. I can see on my runner machine that cl.exe is started and that it locks some files in .conan/data/libiconv directory. If I killl cl.exe, then building continuesand the next lines are:
Then building stucks again, and killing cl.exe does the trick again, i.e., gitlab runner probably re-starts it and continues building. After several such kills libiconv successfully builds. My environment is:
Maybe you can give some me some advice where to look for the fix? Thanks in advance.