search

microsoft / LSKV

A Ledger-backed Secure Key-Value store (LSKV), built on the Confidential Consortium Framework (CCF)
https://microsoft.github.io/CCF
MIT License
34 stars 6 forks source link

Add some custom tests with pytest #150

Closed jeffa5 closed 1 year ago

jeffa5 commented 1 year ago

This mostly sets up the fixtures for testing and getting the sandbox set up.

heidihoward commented 1 year ago

Great, is the plan to add receipts too once merged?

github-actions[bot] commented 1 year ago

Plots from benchmarking at fa647ca2b3cbe2e97aef6b84e4565783cba02834

Available from here

etcd/ecdf-latency_ms-ledger_chunk_bytes-sig_ms_interval-vars.jpg
etcd/scatter-start_ms-latency_ms-ledger_chunk_bytes-snapshot_tx_interval-vars.jpg
etcd/scatter-start_ms-latency_ms-sig_tx_interval-sig_ms_interval-vars.jpg
etcd/throughput_bar-rate-ledger_chunk_bytes-sig_ms_interval-vars.jpg
k6/ecdf-latency_ms-None-None-vars.jpg
k6/scatter-start_ms-latency_ms-None-None-vars.jpg
perf/ecdf-latency_ms-http_version-path-vars.jpg
perf/scatter-start_ms-latency_ms-http_version-path-vars.jpg
ycsb/ecdf-latency_ms-operation-workload-vars.jpg
ycsb/scatter-start_ms-latency_ms-operation-workload-vars.jpg
ycsb/throughput_bar-rate-operation-workload-vars.jpg
github-actions[bot] commented 1 year ago

Plots from benchmarking at a77e2e2d23e878b21628103c3ec2b8d62ee9c2a0

Available from here

etcd/ecdf-latency_ms-ledger_chunk_bytes-sig_ms_interval-vars.jpg
etcd/scatter-start_ms-latency_ms-ledger_chunk_bytes-snapshot_tx_interval-vars.jpg
etcd/scatter-start_ms-latency_ms-sig_tx_interval-sig_ms_interval-vars.jpg
etcd/throughput_bar-rate-ledger_chunk_bytes-sig_ms_interval-vars.jpg
k6/ecdf-latency_ms-None-None-vars.jpg
k6/scatter-start_ms-latency_ms-None-None-vars.jpg
perf/ecdf-latency_ms-http_version-path-vars.jpg
perf/scatter-start_ms-latency_ms-http_version-path-vars.jpg
ycsb/ecdf-latency_ms-operation-workload-vars.jpg
ycsb/scatter-start_ms-latency_ms-operation-workload-vars.jpg
ycsb/throughput_bar-rate-operation-workload-vars.jpg
heidihoward commented 1 year ago

Didn't seem to work for me (same issues as ci): 

================================================= test session starts ==================================================
platform linux -- Python 3.8.10, pytest-7.2.0, pluggy-1.0.0 -- /home/heidi/LSKV/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /home/heidi/LSKV
plugins: anyio-3.6.2
collected 3 items

tests/test_single.py::test_starts ERROR                                                                          [ 33%]
tests/test_single.py::test_kv_latest ERROR                                                                       [ 66%]
tests/test_single.py::test_kv_historical ERROR                                                                   [100%]

======================================================== ERRORS ========================================================
____________________________________________ ERROR at setup of test_starts _____________________________________________
file /home/heidi/LSKV/tests/test_single.py, line 16
  def test_starts(http1_client):
file /home/heidi/LSKV/tests/test_common.py, line 192
  @pytest.fixture(scope="module")
  def http1_client(sbox):
E       fixture 'sbox' not found
>       available fixtures: anyio_backend, anyio_backend_name, anyio_backend_options, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, http1_client, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, sandbox, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

/home/heidi/LSKV/tests/test_common.py:192
___________________________________________ ERROR at setup of test_kv_latest ___________________________________________
file /home/heidi/LSKV/tests/test_single.py, line 25
  def test_kv_latest(http1_client):
file /home/heidi/LSKV/tests/test_common.py, line 192
  @pytest.fixture(scope="module")
  def http1_client(sbox):
E       fixture 'sbox' not found
>       available fixtures: anyio_backend, anyio_backend_name, anyio_backend_options, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, http1_client, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, sandbox, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

/home/heidi/LSKV/tests/test_common.py:192
_________________________________________ ERROR at setup of test_kv_historical _________________________________________
file /home/heidi/LSKV/tests/test_single.py, line 46
  def test_kv_historical(http1_client):
file /home/heidi/LSKV/tests/test_common.py, line 192
  @pytest.fixture(scope="module")
  def http1_client(sbox):
E       fixture 'sbox' not found
>       available fixtures: anyio_backend, anyio_backend_name, anyio_backend_options, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, http1_client, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, sandbox, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

/home/heidi/LSKV/tests/test_common.py:192
=============================================== short test summary info ================================================
ERROR tests/test_single.py::test_starts
ERROR tests/test_single.py::test_kv_latest
ERROR tests/test_single.py::test_kv_historical
================================================== 3 errors in 0.31s ===================================================
make: *** [Makefile:88: tests] Error 1
jeffa5 commented 1 year ago

Great, is the plan to add receipts too once merged?

Definitely!

github-actions[bot] commented 1 year ago

Plots from benchmarking at bd78926d561d468a8a7bf71f801c8690f10d9df0

Available from here

etcd/ecdf-latency_ms-ledger_chunk_bytes-sig_ms_interval-vars.jpg
etcd/scatter-start_ms-latency_ms-ledger_chunk_bytes-snapshot_tx_interval-vars.jpg
etcd/scatter-start_ms-latency_ms-sig_tx_interval-sig_ms_interval-vars.jpg
etcd/throughput_bar-rate-ledger_chunk_bytes-sig_ms_interval-vars.jpg
k6/ecdf-latency_ms-None-None-vars.jpg
k6/scatter-start_ms-latency_ms-None-None-vars.jpg
perf/ecdf-latency_ms-http_version-path-vars.jpg
perf/scatter-start_ms-latency_ms-http_version-path-vars.jpg
ycsb/ecdf-latency_ms-operation-workload-vars.jpg
ycsb/scatter-start_ms-latency_ms-operation-workload-vars.jpg
ycsb/throughput_bar-rate-operation-workload-vars.jpg