search

rerun-io / rerun

Visualize streams of multimodal data. Fast, easy to use, and simple to integrate. Built in Rust using egui.
https://rerun.io/
Apache License 2.0
6.42k stars 313 forks source link

C++ package manager request tracking issue #4579

Open Wumpf opened 10 months ago

Wumpf commented 10 months ago

For the moment we decided to not support additional build systems and package management for C++, since there's simply too many to choose from and this becomes a bit of a maintenance burden that we're not ready for yet.

This ticket collects the various requests that came up so far and what state they're in:

rgolovanov commented 9 months ago

@abey79 , @Wumpf I've started work on conan recipe (#4420 ) in our local conan system and faced a few things. Maybe, we could discuss it here or in discord or have a call. Here are the topics:

C++ SDK provided in assets has good and clear process of integration with just couple lines of code in CMake, but it seems to be very important that own code is compiled along with some Rerun sources (*.c - files). However, the classical package should contain only pre-built libraries (lib or dll) and header files. I tried to create conan package and it contains only hpp files with these libs:

Next, when I try to link rerun conan package to existing project XXXX I'm getting these linker errors - apparently libs don't contain all code:

[build] rerun_c__win_x64.lib(re_sdk_comms-0765e134c12468ee.re_sdk_comms.f78a4b73346f03c4-cgu.05.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(re_sdk-9a5f3c7fa3d85f9c.re_sdk.a3283a595e0d2873-cgu.00.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(re_sdk-9a5f3c7fa3d85f9c.re_sdk.a3283a595e0d2873-cgu.12.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(re_sdk_comms-0765e134c12468ee.re_sdk_comms.f78a4b73346f03c4-cgu.01.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSAStartup [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSACleanup [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_freeaddrinfo [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_shutdown [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSAGetLastError [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_setsockopt [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_getsockopt [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_recv [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSARecv [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSASend [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_send [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_accept [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_ioctlsocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_getpeername [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSADuplicateSocketW [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSASocketW [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_getaddrinfo [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_NtCreateFile [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_RtlNtStatusToDosError [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_NtReadFile [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_NtWriteFile [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_connect [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_select [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_recvfrom [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_BCryptGenRandom [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(getrandom-b857062dfe0c4b33.getrandom.2ae02711c9719d53-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_BCryptGenRandom [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_bind [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_listen [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_getsockname [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_sendto [XXXXXXXXXX.vcxproj]
[build] XXXXX.exe : fatal error LNK1120: 30 unresolved externals [XXXXXXXXXX.vcxproj]

I would like to understand better the structure of C++ code, how to make it compilable into self-sufficient library and how did you get rerun_c__win_x64.lib - what does it contain in practice.

I'm thinking of maybe forcing linker to produce all symbols by adding following options (not sure it will resolve all linker problems though):

MSVC: /WHOLEARCHIVE option in the linker. GCC use the -Wl,--whole-archive linker option.

What do you think?

Wumpf commented 9 months ago

@rgolovanov the missing symbols you hit there are from a bunch of very common system dependencies. These to be accurate ws2_32.dll Bcrypt.dll Userenv.dll ntdll.dll Afaik whole-archive linking is generally discouraged since it can lead to a lot of bloat (everyone else now using anything from the windows system comes with their own winsock & ntdll etc.). I'm not familiar with Conan, but I'd assume that these system libraries would be explicitly specified as a dependency whenever linking with the static version of Rerun.

rgolovanov commented 9 months ago

@Wumpf thanks for comment. Yes, now I see it should come from here but apparently was cutoff during conan package creation. I'll check what can be done 👍

traversaro commented 7 months ago

For what regards C++ SDK support for conda-forge, this is now solved (at least on Linux/macOS) with the librerun-sdk package, see https://github.com/conda-forge/staged-recipes/pull/25648 and https://github.com/conda-forge/librerun-sdk-feedstock .

Wumpf commented 7 months ago

@traversaro awesome! Would you mind creating a PR that adds this to the docs somewhere in here https://github.com/rerun-io/rerun/tree/main/rerun_cpp#build--distribution ?

traversaro commented 7 months ago

@traversaro awesome! Would you mind creating a PR that adds this to the docs somewhere in here https://github.com/rerun-io/rerun/tree/main/rerun_cpp#build--distribution ?

Sure, there are a bit of aspects that I would to tackle before (support for it in the opencv/eigen example, Windows support, general dogfooding it to understand if it works fine), and then I think we can can add it there, does it make sense?

Wumpf commented 7 months ago

Of course, whenever you feel comfortable with it :)

traversaro commented 5 months ago

Of course, whenever you feel comfortable with it :)

Now that we have the Windows package as well, I think it is fine to add it: https://github.com/rerun-io/rerun/pull/6381 .