Build instructions "Cleaner builds" does not allow you to run individual functional tests

[rserranon]

When using:

# Clean current source dir in case it was already configured
make distclean

# Make new build dir
mkdir build && cd build

# Run normal build sequence with amended path
../autogen.sh
../configure --your-normal-options-here
make -j `nproc

test_runner.py is copied to the bitcoin/build/test/funcional directory, but not all the individual functional tests, the file config.ini gets generated in bitcioin/build/test directory, allowing test_runner.py to be run with test/fuctional/test_runner.py from the bitcoin/build directory without any problem.

However, if you want to run the individual functional test, you have to do it from the bitcoin/test/functional directory, but if you do that, it cannot find the config.ini file. Even if you run ../../test/functional/p2p_headers_sync_with_minchainwork.py from the bitcoin/build/test directory, where config.ini is located, it's looking for it on the bitcoin/test subdirectory.

FileNotFoundError: [Errno 2] No such file or directory: '/Users/... .../bitcoin/test/config.ini'

[willcl-ark]

Wow, what a great issue, thanks for reporting!

I've not done it that way myself before, but it makes total sense to do so and I hope there's a clean way to do it... Going to look into it

[willcl-ark]

So it looks like env vars handle this, just testing now...

[rserranon]

Thx, I also find some issues generating Doxygen documentation when using a build subdirectory, I will create a separate issue if I confirm it's not working

[willcl-ark]

Hmmm, it seems to work for me from the build directory?

Could you provide exact reproduction steps? I did:

make distclean
mkdir build && cd build
../autogen.sh; and ../configure BDB_LIBS="-L$BDB_PREFIX/lib -ldb_cxx-4.8" BDB_CFLAGS="-I$BDB_PREFIX/include" --with-sqlite=yes --enable-experimental; and bear -- make -j16
./test/functional/test_runner.py
pwd
test/functional/test_runner.py p2p_ping.py

[rserranon]

I was trying to run the test by directly calling the python file as: test/functional/p2p_ping.py, if I run it through the test_runner.py as you do above it works for me too. The problem is if someone tries to run individual tests by calling directly their python file, as recommended in the trest/README.md in the bitcoin repository, although they also mention how to run them using the test_runner.py as you did.

[willcl-ark]

OK so there are a few solutions here...

Firstly , I think you can still call the tests directly by using the --configfile option on an individual test. So I think this will work:

make distclean
mkdir build && cd build
../autogen.sh
../configure
make -j16
cd ..
test/functional/p2p_ping.py --configfile build/test/config.ini

That said, I've also been following and testing https://github.com/bitcoin/bitcoin/pull/25797 which uses cmake. I have just had a go testing with the new cmake (of ouf source builds) and the following worked with that PR checked out:

To use BDB compiled from depends:

make -j 16 -C depends NO_BOOST=1 NO_LIBEVENT=1 NO_QT=1 NO_SQLITE=1 NO_NATPMP=1 NO_UPNP=1 NO_ZMQ=1 NO_USDT=1 ALLOW_HOST_PACKAGES=1
cmake --toolchain depends/x86_64-pc-linux-gnu/share/toolchain.cmake -S . -B build
cd build
make -j16
cd ..
test/functional/p2p_ping.py --configfile build/test/config.ini

Without BDB even cleaner:

cmake -S . -B build
cd build
make -j16
cd ..
test/functional/p2p_ping.py --configfile build/test/config.ini

So I think what I might do here is to leave this issue open for a while longer, as I think the cmake PR is coming along fast now (despite being in draft still), to remind me to potentially update this section in the future when that lands.

In the mean time I will amend the docs to include how to run individual tests by specifying the config.ini file.

[willcl-ark]

Updated for now in https://github.com/chaincodelabs/onboarding-to-bitcoin-core/commit/e1df4c235e64933208c2bcb417f36e9fdd0bbfd0

[fanquake]

ALLOW_HOST_PACKAGES

Just noting that there should be no need or reason to use ALLOW_HOST_PACKAGES here.

[adamjonas]

closed via e1df4c235e64933208c2bcb417f36e9fdd0bbfd0