frink-okn / FRINKIssues

0 stars 1 forks source link

Finalize testing plan #10

Open cbizon opened 3 months ago

YaphetKG commented 2 months ago

Next step finalizing testing plan is to get some of the sample queries as test cases, Also including plan on configuration of test including things such as

YaphetKG commented 2 months ago

Testing plan is ready (https://docs.google.com/document/d/1mvmPPrHM8wklAlokJbU2f5ncW0EWAuBBIWjRRxpQ9vI/edit#heading=h.o45zfz7vq5zp)

cbizon commented 1 month ago

Are we ready to implement the plan now?

YaphetKG commented 1 month ago

We now have a first round of tests on the predator server (https://predator-load-testing.apps.renci.org/ui/tests) The tests are the cartesian of this queries (https://github.com/frink-okn/api-tests/blob/main/performance-tests/tests.json) with this set of servers (https://github.com/frink-okn/api-tests/blob/main/sparql-servers.json) . This (https://github.com/frink-okn/api-tests/blob/main/create_artillery_tests.py) script can be used to update the tests.

Next steps:

  1. The performance tests might need a more closer look , this are very simple queries , and some of them are most likely going to require parametrization. (@mahir256 , @balhoff ) if you look at them and provide any feedback on these that would be awesome. (https://github.com/frink-okn/api-tests/blob/main/performance-tests/tests.json)
  2. Think about how to schedule these.
  3. Think about setting benchmarks, (running these and settling on a performance we like) and setup notification system, if scheduled runs fall behind the benchmarks (?)
  4. Add tests with regards to Regression as fore mentioned in the testing plan.