REST API add regression/verification script-s/test-s

[mbohun]

we need to create/add regression tests:

• [x] input: create a small "constant" sub-set of tblBiota.csv
  • the "real" tblBiota.csv from NAQS can - in theory - change anytime
  • the "constant" sub-set of tblBiota.csv will be an extract of the "real" tblBiota.csv but maintained by us (it is important to cover as many diff input types/scenarios as possible)
• [x] run/execute these regression test scripts EVERY TIME you want to verify that we:
  • did NOT break something that was working before
  • did NOT change something "unintentionally"
  • detect ALL changes (All => as-many-as-we-can) to avoid surprise-s; sometimes (or even often) a change/difference is NOT automatically a BUG/error, but we still want to see the change/difference to be clearly identified/reported-to-us by the automated regression test-s

DETAILS / NOTES / SUGGESTIONS:

• typical/obvious candidates (for addition of test/regression scripts) are the /ws, /solr endpoints, where a simple regression test script (BASH, curl, jq based) should immediately point out any differences between "the current" and "the previous" version, i.e: something that was working "before" is not working "anymore"/"now"; we do have already number issues related to this problem:

  • 20

  • 19

  • 16

  • 10

• some test-s are rather trivial (HOWEVER they can SAVE US HOURS / DAYS compared to a ad hoc investigation):
  • do i have ANY records in the SOLR at all; NOTE: this test can be (should be) done for each of the kingdom-s too; where a kingdom is one of:
  • A (Animalia)
  • P (Plantae)
  • L (Algae)
  • F (Fungi)
  • B (Bacteria)
  • V (Viruses)
  • the same type of test can be done for each of the BNTi groups/shards:
  • icn: P, L, F
  • iczn: A
  • icnp: B
  • icnvc: V
  • is the total number of records diff? and if yes, how big is the difference?
  • is the number AND the names of JSON fields the same as before, here is a real-life example of this test/comparison: https://gist.github.com/mbohun/b2ec750a338535328e6f803e79b6806e/revisions
  • are the values (for the same record/guid) same as before (for example we know that the "id" is a randomly assigned UUID by SOLR, so a field that is not relevant/subject to regression testing)
• issue #9: "verify generated/imported SOLR data against input CSV (missing records)"
• we should consider integrating the dev/test env with https://mitmproxy.org/ and keep "automatically" record all the REST API communication so we can easily browse/review the history in order to identify a change (for example a change in REST API function, arguments, etc.)

[mbohun]

This issue is PARTLY related-to/overlapping-wth (the "add versioning info") issue-s:

• 13 "expose / add env info and version info to ag-bie site"

• 15 "Add date of last index creation to home page"

[moziauddin]

A testing script is integrated into the ag-bie-solr repo to test the API on every build. Starting this commit.