This adds another sample Jupyter notebook to compare Spark+Parquet and PostgresSQL+views queries.
Note this is on top of #919 so only the second commit belongs to this PR.
E2E test
TESTED:
Ran the pipeline on the large dataset with SQLonFHIR-v2 view-generation enabled. Then built the custom Jupyter docker image and ran queries_large.ipynb.
Description of what I changed
This adds another sample Jupyter notebook to compare Spark+Parquet and PostgresSQL+views queries. Note this is on top of #919 so only the second commit belongs to this PR.
E2E test
TESTED:
Ran the pipeline on the large dataset with SQLonFHIR-v2 view-generation enabled. Then built the custom Jupyter docker image and ran
queries_large.ipynb.Checklist: I completed these to help reviewers :)
[x] I have read and will follow the review process.
[x] I am familiar with Google Style Guides for the language I have coded in.
No? Please take some time and review Java and Python style guides.
[x] My IDE is configured to follow the Google code styles.
No? Unsure? -> configure your IDE.
[ ] I have added tests to cover my changes. (If you refactored existing code that was well tested you do not have to add tests)
[x] I ran
mvn clean packageright before creating this pull request and added all formatting changes to my commit.[x] All new and existing tests passed.
[x] My pull request is based on the latest changes of the master branch.
No? Unsure? -> execute command
git pull --rebase upstream master