Run UAT Rejection Tests

[LJTrent]

Description:

The rejection uat is a python script that runs the auto-analyze against rejected NRs in postgres. It requires use of namex-api and synonyms-api.

https://github.com/bcgov/namex/tree/master/jobs/rejection-uat

This is currently run against the production Namex Postgres database and only includes BC Corps so the query can be tweaked to whatever other entities you want to include.

The original examiner results and the auto-analyze results are stored in a table called like this: uat = UatResults() uat.id = data_dict['id'] uat.nr_num = data_dict['nr_num'] uat.nr_state = data_dict['nr_state'] uat.choice = data_dict['choice'] uat.name = data_dict['name'] uat.name_state = data_dict['name_state'] uat.decision_text = data_dict['decision_text'] uat.conflict_num1 = data_dict['conflict_num1'] uat.conflict1 = data_dict['conflict1'] uat.result_state = data_dict['result_state'] uat.result_decision_text = data_dict['result_decision_text'] uat.result_conflict_num1 = data_dict['result_conflict_num1'] uat.result_conflict1 = data_dict['result_conflict1'] uat.result_response = data_dict['result_response'] uat.result_duration_secs = data_dict['result_duration_secs']

All columns that start with result are the auto-analyze results all other columns are the original examination.

Once the script is run, the results are exported to a spreadsheet and given to Genevieve to review. She reviews any differences in decision. That may entail looking at conflict names to ensure they are still a conflict (not historical), verifying synonyms, word classification, consent words were necessary and providing feedback to Arturo for missing rules.

Of the ones we ran. we did not have any the auto-analyze approved that examiners rejected. We did have ones the we conflicted but examiners approved. The set was 185. It is time-consuming to run the script depending on how common a name is and how many conflicts. we retrieve all conflicts and score them.

We are missing some rules and we also are in the process of tweaking the score.

Details

• [x] make this a job that runs constantly (schedule tbd)
  • [x] uses its own db that contains NRs from namex
  • table for NRs will include pass/fail/untested column, request time, datetime/version/run number
  • check uat db for ones that aren't in it and copy them over
  • table for individual job runs (includes avg request time, number failed, number passed, total NRs tested, job seq number, datetime run, emailed results)
  • [x] updates db with new approved/rejected NRs from prod into its db
  • [x] test against get in bc_name_analysis
  • [x] grabs rejected names in NRs (starting with most recent?) and sees if auto analyzer approves/rejects
    • [x] ignores names from csv in folder
    • [x] stores these + results in table (add columns to help specify different classes of failures?)
    • [x] can add specific names to csv and will run those
  • [x] grabs approved names in NRs (starting with most recent) and sees if auto analyzer approves/rejects how can we do this without it failing against itself every time? -- grab NRs that haven't been examined yet after hours (time > 8pm so none get approved while job runs?) -- these are given a different job type and compared later?
    • [x] grabs up to MAX_ROWS new unexamined NRs (to ensure that it tests all of the ones it brings over in this job run)
    • [x] ignores names from csv in folder
    • [x] stores these + results in table (add columns to help specify different classes of failures?)
    • [x] can add specific names to csv and will run those
• new ticket for below: https://github.com/bcgov/entity/issues/5268
  • separate job to send as spread sheet to genevieve? use emailer code/notify-api?
  • look into stevens notebook reports for this part for running notebook collection on a scheduled job
  • create notebook for gathering results from uat db
    • shows failed and displays NR details, gives avg request time, overall percentages for fail/pass rate
    • saves results similar to existing support notebook and emails it?

[jdyck-fw]

This estimate was likely wrong, instead it would be closer to 25