Provide reproducible experiment and results showing existing data does not meet current needs

MichaelClifford commented 4 years ago

@MichaelClifford has reported that the current RHV dataset stored in s3://DH-PLAYPEN/ccit is incompatible with the existing flake analysis implementation. However, he has provided no methodology or documentation for how he performed these tests or how someone could reproduce his work and verify the results.

The work that was done to come to these conclusions should be included in this repo and include sufficient documentation that it can be easily reproduced by another data scientists.

Acceptance Criteria:

[x] Include the scripts and notebooks used to perform data validation tests.
[x] Include clear and sufficient documentation to run the scripts, reproduce the tests and verify (hopefully) the results.

MichaelClifford commented 4 years ago

See #9 it should address this issue.

@Shreyanand let me know if you think that PR is sufficient. I could also add some documentation on how we determined the existing data requirements if you think its needed. Let me know : )

Shreyanand commented 4 years ago

@MichaelClifford The notebook looks sufficient. We could probably update the development notes / README with results from that notebook.

MichaelClifford commented 4 years ago

This issue is closed by #9

aicoe-aiops / flake-qe

Provide reproducible experiment and results showing existing data does not meet current needs #8