Open ALEXRR1005 opened 3 years ago
This is a fairly long answer with lots of links, but you did ask a very general question. It's also a very good question.
are you using the LumoSQL library with test programs, or with the commandline interface?
We run a multiway build, test and benchmarking matrix to present the complexity as simply as possible. There's a lot of variables. Even in vanilla SQLite (which LumoSQL supports 100% for build, test and execution) there are way more variables than anyone can hold in their head, and before LumoSQL there was no tool to compare them.
If you just want to get going, then type this:
$ make what
That will list all of the possibilities currently available to it. You will need to first need to install the LumoSQL dependencies of course, but there are very few and it is easy.
To answer your question very directly, LumoSQL "make benchmark" uses the commandline interface, in the same way that it does with SQLite. We do not yet run tests with the library linked to other programs although that is part of what we are working on now (see the Phase II description. ) We recognise there is a difference between benchmarking and testing, and we are trying to make that increasingly clear in the many benchmark and test routines we are writing.
"make benchmark" does multiple runs and stores the result in a standard SQLite database, along with a per-run GUID and metadata information identifying the environment and codebase as accurately as possible, so that dozens or hundreds of people can aggregate their results in a single database. This database can then be queried using the benchmark-filter tool.
Here is an extract from the detailed documentation with an example use of the benchmarking code:
...to build two versions of plain sqlite3, two versions of sqlite3+LMDB and one version of BDB with its own sqlite3:
tclsh tool/build.tcl build not-fork.d /tmp/objects \
SQLITE3_VERSION='3.14.15 3.33.0' \
USE_LMDB=yes LMDB_VERSIONS='0.9.9 0.9.27' SQLITE_FOR_LMDB=3.8.0 \
USE_BDB=yes BDB_STANDALONE='18.1.32'
To add option debug=on to the build:
tclsh tool/build.tcl build not-fork.d /tmp/objects myresults.sqlite \
SQLITE3_VERSION='3.14.15 3.33.0' \
USE_LMDB=yes LMDB_VERSIONS='0.9.9 0.9.27' SQLITE_FOR_LMDB=3.8.0 \
USE_BDB=yes BDB_STANDALONE='18.1.32' \
-DEBUG=on
Which backend are you using?
All of them, including the default SQLite backend. By selecting "ALL" or various subsets, "make benchmark" can compare different versions of SQLite, or SQLite vs LumoSQL with particular backends.
Does this help?
Describe the bug
A clear and concise description: are you using the LumoSQL library with test programs, or with the commandline interface? Which backend are you using?
To Reproduce
Steps to reproduce the behaviour:
Expected behaviour
A clear and concise description of what you expected to happen.
Error cut/paste
Show the terminal output demonstrating the error
Additional context:
OS: [e.g. Debian Linux 10.5, or FreeBSD 12.1R] Tools: are you using non-default tools? Don't include
configure
unless you think it is relevant.