[boost] boost/1.7?.0: Windows cannot use ICU as i18n backend

datalogics-robb commented 3 years ago

Boost can be configured to use the ICU library for unicode support in locale and regex, and I have been able to get that configuration working on Mac, Linux, Solaris and AIX, but no combination of shared/static runtime, shared or static libraries, debug or release builds will configure boost to use ICU.

A stack overflow question hints at the reason why: https://stackoverflow.com/questions/12142571/building-and-using-boost-for-windows-visual-studio-with-a-custom-icu

when it mentions that both debug and release libs and DLLs need to be copied into the icu distribution folder.

To reproduce, it is also important to remove the previous configuration run's cache project-cache.jam and then to examine the output saved to config.log in the same directory.

This project should probably attempt to build such a configuration in CI to verify that it is broken, or to verify when it is fixed.

Package and Environment Details (include every applicable attribute)

Package Name/Version: boost/1.72.0 through boost/1.75.0
Operating System+version: Windows 10
Compiler+version: Visual Studio 2017
Conan version: conan 1.34.0
Python version: Python 3.8.5

Conan profile (output of `conan profile show default` or `conan profile show <profile>` if custom profile is in use)

[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.cppstd=14
compiler.runtime=MD
compiler.version=15
os=Windows
os_build=Windows
[options]
booost:i18n_backend=icu
boost:debug_level=4
boost:extra_b2_flags=--hash
boost:i18n_backend=icu
boost:pch=False
boost:shared=False
boost:without_context=True
boost:without_contract=True
boost:without_coroutine=True
boost:without_fiber=True
boost:without_graph=True
boost:without_iostreams=True
boost:without_log=True
boost:without_math=True
boost:without_program_options=True
boost:without_random=True
boost:without_test=True
boost:without_timer=True
boost:without_type_erasure=True
boost:without_wave=True
icu:shared=True

Steps to reproduce (Include if Applicable)

Use option -o i18n_backend=icu when building boost, and verify the boost b2 check for ICU fails and reports no icu used.

Logs (Include/Attach if Applicable)

Click to expand log

``` file C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu_obj.obj.rsp "libs\locale\src\..\build\has_icu_test.cpp" -Fo"C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu_obj.obj" -TP /Z7 /Od /Ob0 /W3 /GR /MDd /Zc:forScope /Zc:wchar_t /favor:blend /wd4675 /EHs /std:c++14 -c -DBOOST_ALL_NO_LIB=1 "-I." "-IC:\Users\robb\.conan\data\icu\68.2\_\_\package\611fcefb561f89074ece4ede322b83bfae850b30\include" compile-c-c++ C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu_obj.obj call "C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\standalone\msvc\1516fc4ab8d17a0765a1f78ddebd56c2\msvc-setup.bat" >nul cl /Zm800 -nologo @"C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu_obj.obj.rsp" has_icu_test.cpp file C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu.exe.rsp "C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu_obj.obj" "icudt.lib" "icuind.lib" "icuucd.lib" msvc.link C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu.exe call "C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\standalone\msvc\1516fc4ab8d17a0765a1f78ddebd56c2\msvc-setup.bat" >nul link /NOLOGO /INCREMENTAL:NO /DEBUG /MACHINE:X64 /MANIFEST /subsystem:console /out:"C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu.exe" /LIBPATH:"C:\Users\robb\.conan\data\icu\68.2\_\_\package\611fcefb561f89074ece4ede322b83bfae850b30\lib" @"C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu.exe.rsp" if %ERRORLEVEL% NEQ 0 EXIT %ERRORLEVEL% LINK : fatal error LNK1181: cannot open input file 'icuind.lib' ...failed msvc.link C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu.exe C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu.pdb... ...failed updating 2 targets... ```

Performing configuration checks

    - default address-model    : 32-bit
    - default architecture     : x86
    - compiler supports SSE2   : yes
    - compiler supports SSE4.1 : yes
    - has synchronization.lib  : yes
    - has BCrypt API           : yes
    - has stat::st_mtim        : no
    - has stat::st_mtimensec   : no
    - has stat::st_mtimespec   : no
    - has stat::st_birthtim    : no
    - has stat::st_birthtimensec : no
    - has stat::st_birthtimespec : no
    - has statx                : no
    - has statx syscall        : no
    - cxx11_alignas            : yes
    - cxx11_constexpr          : yes
    - cxx11_decltype           : yes
    - cxx11_hdr_tuple          : yes
    - cxx11_template_aliases   : yes
    - cxx11_variadic_templates : yes
    - icu                      : no
    - icu (lib64)              : no
    - cxx11_defaulted_functions : yes
    - cxx11_noexcept           : yes
    - cxx11_rvalue_references  : yes
    - cxx11_static_assert      : yes
    - std::fstream is moveable and swappable : yes
    - Has Large File Support   : yes
    - has_icu builds           : no
    - libbacktrace builds      : no
    - addr2line builds         : no
    - WinDbg builds            : yes
    - WinDbgCached builds      : yes

Component configuration:

    - atomic                   : building
    - chrono                   : building
    - container                : building
    - context                  : not building
    - contract                 : not building
    - coroutine                : not building
    - date_time                : building
    - exception                : building
    - fiber                    : not building
    - filesystem               : building
    - graph                    : not building
    - graph_parallel           : not building
    - headers                  : not building
    - iostreams                : not building
    - json                     : building
    - locale                   : building
    - log                      : not building
    - math                     : not building
    - mpi                      : not building
    - nowide                   : building
    - program_options          : not building
    - python                   : not building
    - random                   : not building
    - regex                    : building
    - serialization            : building
    - stacktrace               : building
    - system                   : building
    - test                     : not building
    - thread                   : building
    - timer                    : not building
    - type_erasure             : not building
    - wave                     : not building

datalogics-robb commented 3 years ago

A release build of ICU on windows has these libraries under lib:

 dir C:\Users\robb\.conan\data\icu\68.2\_\_\package\611fcefb561f89074ece4ede322b83bfae850b30\lib\
 Volume in drive C has no label.
 Volume Serial Number is 4E91-0A89

 Directory of C:\Users\robb\.conan\data\icu\68.2\_\_\package\611fcefb561f89074ece4ede322b83bfae850b30\lib

03/12/2021  01:27 PM    <DIR>          .
03/12/2021  01:27 PM    <DIR>          ..
03/12/2021  01:27 PM             1,686 icudt.lib
03/12/2021  01:27 PM         2,776,692 icuin.lib
03/12/2021  01:27 PM            12,584 icuio.lib
03/12/2021  01:27 PM            37,330 icutest.lib
03/12/2021  01:27 PM            70,118 icutu.lib
03/12/2021  01:27 PM         1,029,402 icuuc.lib
               6 File(s)      3,927,812 bytes
               2 Dir(s)  243,011,018,752 bytes free

SpaceIm commented 3 years ago


"C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu_obj.obj"    
"icudt.lib" 
"icuind.lib" 
"icuucd.lib"
msvc.link C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu.exe

        call "C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\standalone\msvc\1516fc4ab8d17a0765a1f78ddebd56c2\msvc-setup.bat"  >nul
 link /NOLOGO /INCREMENTAL:NO /DEBUG /MACHINE:X64 /MANIFEST /subsystem:console /out:"C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu.exe" /LIBPATH:"C:\Users\robb\.conan\data\icu\68.2\_\_\package\611fcefb561f89074ece4ede322b83bfae850b30\lib"  @"C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu.exe.rsp"
        if %ERRORLEVEL% NEQ 0 EXIT %ERRORLEVEL%

LINK : fatal error LNK1181: cannot open input file 'icuind.lib'
...failed msvc.link C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu.exe C:\Users\robb\.conan\data\boost\1.73.0\_\_\build\491db9cc8564376c4db3d2f68ad28bbc2648a5a4\boost\bin.v2\libs\locale\build\aa5f60dfd728cc291a7e1357df68268d\has_icu.pdb...
...failed updating 2 targets...

Is it the log of your Release build? On Windows, all icu libs has d suffix in Debug build only (and s prefix if static). It's weird to see LINK : fatal error LNK1181: cannot open input file 'icuind.lib'

datalogics-robb commented 3 years ago

Yes - you can see from that list, the libs specified are a mixture of debug: icuind.lib icuucd.lib and release icudt.lib I am working on a patch (and understanding b2/bjam) but so far haven't gotten it working in a Release build.

datalogics-robb commented 3 years ago

https://stackoverflow.com/questions/12142571/building-and-using-boost-for-windows-visual-studio-with-a-custom-icu

datalogics-robb commented 3 years ago

Further testing seems to show that when only --with-locale is passed, the ICU build is found (when the bin dir is in the PATH and -sICU_PATH is passed on the b2 command line), but that when ---with-regex is added, it fails to build the has_icu.exe test program.

datalogics-robb commented 3 years ago

Is there a way I can specify that the libs are not in lib64 (and dll's not in bin64)? It seems like that may correct the problem.

...updating 8 targets...
compile-c-c++ C:\Users\robb\boost_build\boost\bin.v2\libs\regex\build\msvc-14.2\release\link-static\threading-multi\has_icu_test.obj
has_icu_test.cpp
msvc.link C:\Users\robb\boost_build\boost\bin.v2\libs\regex\build\msvc-14.2\release\link-static\threading-multi\has_icu.exe
LINK : fatal error LNK1181: cannot open input file 'icudt.lib'

        call "C:\Users\robb\boost_build\boost\bin.v2\standalone\msvc\msvc-14.2\msvc-setup.bat"  >nul
 link /NOLOGO /INCREMENTAL:NO "C:\Users\robb\boost_build\boost\bin.v2\libs\regex\build\msvc-14.2\release\link-static\threading-multi\has_icu_test.obj"    "icudt.lib" "icuin.lib" "icuuc.lib"  /MACHINE:X64 /MANIFEST:EMBED /subsystem:console /out:"C:\Users\robb\boost_build\boost\bin.v2\libs\regex\build\msvc-14.2\release\link-static\threading-multi\has_icu.exe" /LIBPATH:"c:\Users\robb\.conan\data\icu\68.2\datalogics\stable\package\85e67157cc2016cd6153bf94a74236aed73d91ca\bin64" /LIBPATH:"c:\Users\robb\.conan\data\icu\68.2\datalogics\stable\package\85e67157cc2016cd6153bf94a74236aed73d91ca\lib64"

SSE4 commented 3 years ago

I've took a look, and unfortunately, it seems to be very broken in boost itself

studying boost.locale, by default it searches some hard-coded library names for ICU (e.g. icudt). however, for static ICU build, the name is sicudt. there is an option to use custom link-flags whatever you want for ICU : ICU_LINK. however, it's incompatible with boost.regex:

error: The ICU_LINK option is no longer supported by the Boost.Regex build - please refer to the documentation for equivalent options

as it has newer approach to specify library names: ICU_ICUUC_NAME, ICU_ICUDT_NAME, ICU_ICUIN_NAME. and this is not supported by boost.locale. it's very unfortunate boost.locale and boost.regex use different and mutually exclusive ways to configure ICU linking...

in other words, we're running into the issue https://github.com/boostorg/locale/issues/55

I think it will require at very least new PR to boost::locale module to apply changes similar to https://github.com/boostorg/regex/commit/fc4dc17dc745a9fc4ab1849cef3710354a6b7782

/cc @grafikrobot

SSE4 commented 3 years ago

okay, I went ahead and opened PR: https://github.com/boostorg/locale/pull/70 if you are willing to help, and get that available in the next release (boost 1.78):

review PR providing your feedback (I am not an expert in b2, boost jam and boost build)
test PR locally and provide with your results

once it's merged into boost, we can prepare PR for conan recipe to support new variables

Cogitri commented 2 years ago

Unfortunately it seems like this is still broken - is there something that can be done to help get this resolved?

datalogics-robb commented 2 years ago

Sorry, I had completely forgotten that this PR existed.

@SSE4 I don't think really anyone is an expert in b2, boost jam and boost build - which is why I was hoping for the CMake build to come to fruition, but that seems to still be more of a novelty. I'm afraid we don't have the resources to commit to this.

SSE4 commented 2 years ago

@datalogics-robb thee only b2 / boost jam / boost build expert I know is @grafikrobot things you can do to get this resolved faster:

write on boost mail list
write on #boost channel (cpplang slack)
write on boost github issues (https://github.com/boostorg/locale/issues/55, https://github.com/boostorg/locale/pull/70), or open a new one

in general, more noise we do about it on various communication channels - more chances boost developers realize it's important and prioritize it maybe there are other channels you can use to raise the attention - twitter, reddit, or whatever solical network could make a hype about it

Cogitri commented 2 years ago

FWIW, it appears that this "only" affects icu+boost setups where icu:shared=False || icu:build_type=Debug && boost:build_type=Debug. For the time being I'm building both icu and boost shared and with build_type=Release as workaround.

I've also started a thread on the cpplang slack.

conan-io / conan-center-index