Test environment (using `conan`)

aarlt commented 2 years ago

Same as https://github.com/crossbario/autobahn-cpp/pull/234 but using Conan as package manager.

[X] basic integration
[ ] adapt GitHub workflow

aarlt commented 2 years ago

At least so far I can tell that Conan is also very nice. However, the interesting part will be the setup of the GitHub workflows, but theoretically there should not be any major problem - but lets see.

aarlt commented 2 years ago

Build instructions for this PR:

# checkout
git clone git@github.com:aarlt/autobahn-cpp.git
cd autobahn-cpp
git checkout test-environment-conan

# build directory
mkdir build
cd build

# initialise conan
conan install --build missing ../conanfile.py

# build
cmake ..
make

# test (if docker daemon is running)
make test

aarlt commented 2 years ago

Theoretically we could even support conan and vcpkg at the same time: the user could just select it's favourite package manager, where let's say conan is default.

aarlt commented 2 years ago

Maybe https://github.com/conan-io/cmake-conan is very interesting, because it seem to allow using conan without manually running conan before cmake invocation.

oberstet commented 2 years ago

oh, this is fantastic!! really cool that you give conan a try, and if it works with vcpkg at the same time - well, that's what I'd call "best of both worlds";)

gimme a bit time to digest it, try it etc, I post here!

first need to get over this:

CMake 3.21 or higher is required.  You are running version 3.18.5

cmake/focal,now 3.16.3-1ubuntu1 amd64  [installiert]
  Plattformübergreifendes, quelloffenes Make-System

(base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.4 LTS
Release:    20.04
Codename:   focal

comfort docs for our readme then:

wget https://github.com/Kitware/CMake/releases/download/v3.22.2/cmake-3.22.2-linux-x86_64.sh
chmod +x cmake-3.22.2-linux-x86_64.sh
./cmake-3.22.2-linux-x86_64.sh
echo "export PATH=${HOME}/cmake-3.22.2-linux-x86_64/bin/:${PATH}" >> ~/.profile

oberstet commented 2 years ago

awesome! it generates the cmake build files

websocketpp/0.8.2: Package installed 5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
websocketpp/0.8.2: Downloaded package revision 0
bzip2/1.0.8: Appending PATH environment variable: /home/oberstet/.conan/data/bzip2/1.0.8/_/_/package/da606cf731e334010b0bf6e85a2a6f891b9f36b0/bin
conanfile.py (autobahn-cpp/v20.8.1): Generator txt created conanbuildinfo.txt
conanfile.py (autobahn-cpp/v20.8.1): Generator cmake created conanbuildinfo.cmake
conanfile.py (autobahn-cpp/v20.8.1): Aggregating env generators
conanfile.py (autobahn-cpp/v20.8.1): Generated conaninfo.txt
conanfile.py (autobahn-cpp/v20.8.1): Generated graphinfo
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$

only other thing I had to do: set compiler.libcxx=libstdc++11 in $HOME/.conan/profiles/default

because

(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$ conan install --build missing ../conanfile.py
Auto detecting your dev setup to initialize the default profile (/home/oberstet/.conan/profiles/default)
Found gcc 9
Found clang 10.0
gcc>=5, using the major as version

************************* WARNING: GCC OLD ABI COMPATIBILITY ***********************

Conan detected a GCC version > 5 but has adjusted the 'compiler.libcxx' setting to
'libstdc++' for backwards compatibility.
Your compiler is likely using the new CXX11 ABI by default (libstdc++11).

If you want Conan to use the new ABI for the default profile, run:

    $ conan profile update settings.compiler.libcxx=libstdc++11 default

Or edit '/home/oberstet/.conan/profiles/default' and set compiler.libcxx=libstdc++11

************************************************************************************

....

Installing (downloading, building) binaries...
ERROR: There are invalid packages (packages that cannot exist for this configuration):
botan/2.19.1: Invalid ID: Using Botan with GCC >= 5 on Linux requires "compiler.libcxx=libstdc++11"
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$

now, to build, I had to add a env var like

cmake -DCMAKE_BUILD_TYPE=Release ..

and it builds ... without any hickups ... I can't believe it. with all the deps, this is a pretty big and complex code base. really great work!

...
Requirements
    boost/1.78.0 from 'conancenter' - Cache
    botan/2.19.1 from 'conancenter' - Cache
    bzip2/1.0.8 from 'conancenter' - Cache
    catch2/2.13.8 from 'conancenter' - Cache
    libbacktrace/cci.20210118 from 'conancenter' - Cache
    msgpack-cxx/4.0.3 from 'conancenter' - Cache
    openssl/3.0.1 from 'conancenter' - Cache
    websocketpp/0.8.2 from 'conancenter' - Cache
    zlib/1.2.11 from 'conancenter' - Cache
...
-- Configuring done
-- Generating done
-- Build files have been written to: /home/oberstet/scm/3rdparty/autobahn-cpp/build
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$ 
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$ 
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$ make
[  3%] Building CXX object examples/CMakeFiles/examples_parameters.dir/parameters.cpp.o
...
[ 83%] Building CXX object test/CMakeFiles/test_pubsub.dir/pubsub/pubsub.cpp.o
[ 87%] Building CXX object test/CMakeFiles/test_pubsub.dir/test_main.cpp.o
[ 90%] Linking CXX executable ../bin/test_pubsub
[ 90%] Built target test_pubsub
[ 93%] Building CXX object test/CMakeFiles/test_calls.dir/calls/calls.cpp.o
[ 96%] Building CXX object test/CMakeFiles/test_calls.dir/test_main.cpp.o
[100%] Linking CXX executable ../bin/test_calls
[100%] Built target test_calls
(cpy39_1) (base) oberstet@intel-nuci7:~/scm/3rdparty/autobahn-cpp/build$

aarlt commented 2 years ago

Nice! I still need to fix the windows build (see https://github.com/aarlt/autobahn-cpp/runs/5355028210?check_suite_focus=true) ;) I will try to fix it when I find some time.

Beside the build system stuff, the test environment can be found in test/.

BTW I was trying to build crossbar for windows (because I was not able to find a way to run docker images in windows with GitHub workflow) - but it was not just working, sadly I forgot what was the exact issue there. Using Python & crossbar would be in general the other alternative - to not use docker. However, I think at least for Linux & macOS the usage of the docker images is the best solution. But I also think that it would be nice if we could find a way to also run the test cases on windows.

oberstet commented 2 years ago

BTW I was trying to build crossbar for windows

very brave;) honestly, it's long ago I did that. it is possible, but only for masochists, and also pointless. just using docker is a lot better. use windows as a glorified docker driver is good enough.

here is an example how that can be used in CI / GH actions:

https://github.com/crossbario/autobahn-js/blob/master/.github/workflows/main.yml https://github.com/crossbario/autobahn-js/blob/master/.crossbar/config.json

oberstet commented 2 years ago

rgd test strategy vs a running router, as part of CI, "end to end integration tests", just as some background/details:

over the years, we've developed and tried different approaches in autobahn-python, autobahn-js, autobahn-java and crossbar itself.

the approach in autobahn-js is probably the simplest. the tests

will start crossbar via docker
then tests are driven from https://github.com/crossbario/autobahn-js/blob/master/packages/autobahn/test/test.js
each of these test will open multiple WAMP connections (from one process), do WAMP stuff, and then log messages to stdout.
those messages are designed to indicated that sth that should happen actually did happen, and the messages are "fixed" (don't have changing timestamps or similar)
the stdout of each test is the diff'ed against a "known good" file, eg https://github.com/crossbario/autobahn-js/blob/master/packages/autobahn/test/test_pubsub_basic.txt

the test passes, if the log matches known good exactly.

now, this works and does indeed catch many issues, and in a way positively proves that the basics are working.

however, it obviously also has shortcomings.

how do you initially come up with a "known good" file? how do you prevent testing crossbar specific implementation behavior vs WAMP spec only? how do you test what should not happen (since then there is no message logged ..) ...

but the upside is: it is a straight-forward test strategy that is easy to implement;)

if you are interested, I can explain the approaches in the other repos ... the stuff in crossbar is the most advanced. it contains tests to configure and boot a cluster of router nodes with management node and all the shit. however, for the client libraries testing, this is overkill.

the client lib test strategy IMO should have one important feature: simple and "just works". not introducing even more problems, unrelated to the client library itself.

crossbario / autobahn-cpp

Test environment (using `conan`) #235