Create causal pathway testing infrastructure

Display-Lab / knowledge-base

Precision Feedback Knowledge Base

https://display-lab.github.io/knowledge-base/

Creative Commons Attribution Share Alike 4.0 International

1 stars 1 forks source link

Create causal pathway testing infrastructure #165

Closed mackgalante closed 1 year ago

mackgalante commented 1 year ago

With full implementation of causal pathways, need to have the ability to test their functionality individually, and rapidly at volume.

Make causal-pathway specific input messages (can adapt from vignette data)
Share input_messages for individual postman testing
Allow LDT to test new input_message collection in sequence

Action Items:

[x] Make branch of KB repo
[x] Generate input_message files that trip one of each causal pathway
[x] Host messages in dev branch
[x] Implement and debug causal_pathway automated test from branched input_messages
[x] Release LDT with added functionality in new version

mackgalante commented 1 year ago

Note: have assigned issue to knowledgebase d/t majority of work taking place there. LDT changes will also impact PFP repo, but GitHub doesn't let an issue be assigned to both repos :)

mackgalante commented 1 year ago

Knowledgebase branched, see branch: https://github.com/Display-Lab/knowledge-base/tree/Causal-Pathway-Testing-Infrastructure

mackgalante commented 1 year ago

Input message files created.

Generated from input_message files, formatted so all have staff_ID 7, no further changes.

Uploading to branch shortly.

mackgalante commented 1 year ago

Uploaded to branch, pull request drafted ( #166 )

Will merge to main once LDT is operational

Might require some substantial work refactoring to re-use repoTest code with a new dictionary of files to pull from...

mackgalante commented 1 year ago

Leakdown tester additional features finished, testing and debugging

mackgalante commented 1 year ago

Ran into an issue with using the vignette data - the data is too 'real', there are too many valid causal pathways that can be deemed acceptable for most if not all of the data sets.

Solution: re-craft all xx_cptest.json files to ONLY trip one causal pathway by adjusting both performance data as well as comparator values.

Goal pathways have all social comparators set to minimum or maximums, case-dependent
Social pathways have goal comparators set to max/min as appropriate

Will continue editing files locally, push via upload or by source control.

mackgalante commented 1 year ago

Rev 2 of cptest.json files uploaded.

mackgalante commented 1 year ago

Merged causal pathway testing infrastructure to main, see #166

mackgalante commented 1 year ago

LDT Version 1.6.0 released. See #65 in precision-feedback-pipeline, hash below: https://github.com/Display-Lab/precision-feedback-pipeline/commit/ddc1103484f83f79039895c9eedf39c8eb11a6dc

mackgalante commented 1 year ago

Issue resolved; infrastructure added, tested, and released.