cubed-dev / cubed-benchmarks

Automated benchmark suite for testing the performance of cubed
Apache License 2.0
2 stars 0 forks source link

Write result duration into local database #1

Closed TomNicholas closed 8 months ago

TomNicholas commented 8 months ago

Records the duration time of the one test to a local database.

Doesn't actually work yet - if you run pytest --benchmark it gets as far as creating a benchmark.db file locally before erroring with

E       sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: test_run
TomNicholas commented 8 months ago

This actually works now - you can run it with the benchmark, and then see the results with

from sqlalchemy import create_engine, text
from sqlalchemy.orm import sessionmaker

# Connect to the SQLite database
engine = create_engine('sqlite:///benchmark.db')

# Create a session
Session = sessionmaker(bind=engine)
session = Session()

# Query the existing table and print the results
query = text("SELECT * FROM test_run")
result = session.execute(query)
records = result.fetchall()

for record in records:
    print(record)

# Close the session
session.close()

which gives

(1, '30b586c2259e425ab833d87ae235f49c', 'test_quad_means[configs/local_single-threaded.yaml]', 'test_quad_means', 'benchmarks/test_array.py', None, None, None, '0.13.0', '2024.1.1.dev31+gdb9e448a.d20240203', '0.0.4', '3.1.2', '3.12.0.final.0', 'darwin', None, '2024-03-01 18:39:24.832665', '2024-03-01 18:39:26.787146', 1.9544808864593506, None, None, None, None, None, None)
(2, 'fde11a6e3d204982a3460ac3a7881acb', 'test_quad_means[configs/local_single-threaded.yaml]', 'test_quad_means', 'benchmarks/test_array.py', None, None, None, '0.13.0', '2024.1.1.dev31+gdb9e448a.d20240203', '0.0.4', '3.1.2', '3.12.0.final.0', 'darwin', None, '2024-03-01 18:43:49.368711', '2024-03-01 18:43:51.334510', 1.96579909324646, None, None, None, None, None, None)

Here you can see two entries in the test_run table, including timestamps for measuring test duration (i.e. the test_quad_means test took 1.95 then 1.96 seconds to run).

TomNicholas commented 8 months ago

(There is a commit I forgot to push before merging this - see main for an actually-working version)