vaticle / typedb-benchmark

TypeDB Simulation and Benchmarking Library
GNU Affero General Public License v3.0
9 stars 15 forks source link

Check connections to all DB servers before running any simulations #140

Open james-whiteside opened 1 year ago

james-whiteside commented 1 year ago

Currently, if I run the TypeDB vs Neo4J comparison test without proper authentication for Neo4J, the TypeDB simulation is run in its entirety before the client attempts (and fails) to connect to the Neo4J server.

This could be problematic in the future as the scope of the simluations is upscaled. E.g. I want to run a massive day-long simulation to compare five databases. The first three databases are succesfully connected to and their simulations run, but for whatever reason, a connection cannot be established to the fourth database. The test crashes, and I've now lost three days for what is likely a five-minute fix.

This can be avoided by having the client attempt to establish a connection to each database in the test before running any simulations. That way, any connection errors stop the test immediately before time is potentially wasted. This trial connection should probably be separate to the connection actually used for the test: in the event of very long tests, the trial connection may time out by the time the test has run and so shouldn't be assumed to have persisted.