search

biothings / bte_regression_test

0 stars 1 forks source link

Added GH Action to run manually and periodically #7

Closed ariutta closed 3 years ago

ariutta commented 3 years ago

Hi @kevinxin90, I have a proof of concept GH Action here that pulls in the Docker image, runs every query template in src/query_templates and saves the full_results.csv file as a GH Action artifact.

There are actually two GH Actions: bte_regression_test.yml and bte_regression_test_quick_demo.yml. The second one is just to allow for quick testing of the action itself, running it for just the first two lines of OOPD-22-02-21.csv. To make that work, I added a limit argument with a default of infinity (run for all lines). These changes are not really needed, unless you want to be able to do quick runs just to test the action.

I think you can squash my commits when you merge the pull request (keep the git log easy to read).

ariutta commented 3 years ago

The reason I needed the quick demo is that the full run takes longer than the GH Actions limit of 6 hours. The quick demo takes about an hour.

ariutta commented 3 years ago

We discussed these questions on our call:

  1. Do we want a single combined job, or do we want to split up each input file / query template combo into separate jobs?
  2. For analyzing the data, we can launch a Jupyter notebook and download the relevant artifacts (example). Is that good, or would it be better to make the results immediately viewable online.

We decided for 2) that we will create a static page for summary stats over time + use a Jupyter notebook for interactive analysis of the data.