Avx2 and sse4 optimization doesn't compile in linux system

[parsevalbtc]

Ran the following for compiling libbitcoin node on a Linux System:

CFLAGS="-O3" CXXFLAGS="-O3" ./install.sh --prefix=/home/user --build-boost --enable-ndebug --disable-shared --enable-avx2 --enable-sse4

Commits of the repositories involved are as follows:

libbitcoin-node$ git log
commit 0e7853934e778f7ba03e74b887b6a90c004596d1 (grafted, HEAD -> master, origin/master, origin/HEAD)
Author: Eric Voskuil <eric@voskuil.org>
Date:   Wed Jun 12 21:02:09 2024 -0400

libbitcoin-system$ git log
commit b619ff2dd6c2e77679ce935cb84a177fbb7ae918 (grafted, HEAD -> master, origin/master, origin/HEAD)
Author: Eric Voskuil <eric@voskuil.org>
Date:   Thu Jun 13 21:12:39 2024 -0400

libbitcoin-network$ git log
commit 19177766c968917e3d38306e84f0d7c398d5c7e6 (grafted, HEAD -> master, origin/master, origin/HEAD)
Author: Eric Voskuil <eric@voskuil.org>
Date:   Thu Jun 6 18:04:51 2024 -0400

libbitcoin-database$ git log
commit ed3e0a26b7072266d8b3b81134497ea9f575525b (grafted, HEAD -> master, origin/master, origin/HEAD)
Author: Eric Voskuil <eric@voskuil.org>
Date:   Thu Jun 13 00:47:13 2024 -0400

Compilation went smoothly till the end without particular warnings. Avx2 and sse4 optimizations have been detected in the compilation stream as well:

...
********************** Preparing to build libbitcoin-system **********************
...
checking --enable-avx2 option... yes
checking --enable-avx512 option... no
checking --enable-sse4 option... yes
...

However checking the software with ./bn --hardware command the two optimizations seems to not have been compiled:

$ ./bn -c ../libbitcoin.conf -d
Intrinsics...
arm..... platform:0.
intel... platform:1.
avx512.. platform:0 compiled:0.
avx2.... platform:1 compiled:0.
sse41... platform:1 compiled:0.
shani... platform:0 compiled:0.
neon.... platform:0 compiled:0.

Any idea?

[evoskuil]

I suspect the options are not enabled on the node (or database or network) project. In this case they would be compiled into system and therefore fully utilized unless functions that use them are compiled from templates directly into the node compilation unit. The symbols in the command line tool are read from a system header, which will produce a different result in node if the applicable WITH_... symbol is not defined there. This is the case in the MSVC builds, but may not be in the others. So again, in this case you are likely getting the behavior, just not the recognition in the command line tool.

[evoskuil]

It is designed this way because a good amount of the code relies on templates which use these symbols. The templates will not have the optimization if the symbols are not defined within the compilation unit, so the command line tool catches that possibility. However it may be the case that some or all functions in use are compiled into the system object lib due to the fact that they are wrapped with helper functions and those are typically what is used. But again, not assured.

[evoskuil]

@pmienk - can you make sure that these options are picked up in each of the libs, so that there is no chance that the optimizations will be missed downstream?

[parsevalbtc]

Thanks @pmienk. Will be trying your modifications from the merge.