Update API database using OpenPedCan-analysis v11 data release

[logstar]

After OpenPedCan-analysis v11 data release, update API database using v11 data with the following steps:

• Update OpenPedCan-analysis submodule to v11 release commit.
• Rerun ./db/build_db.sh.
• If any assertion fails, handle the failed assertion.
• If EFO ID changes, update tests/r_test_scripts/test_endpoint_http.R to use the updated EFO IDs.
• Load updated database to local API database server, with DB_LOCATION=local docker-compose up.
• Run tests/run_tests.sh locally.
• Push local changes to this repository, so that the local changes with be deployed to remote DEV API server.
• Deploy the updated database to DEV server for evaluation.
• Create a PR for the changes.

Two branches in this repository are currently deployed remotely.

• de-heatmap branch is deployed on remote DEV API and database server for reviewing https://github.com/PediatricOpenTargets/OpenPedCan-api/pull/60.
• main branch, i.e., release v0.4.0-beta, is deployed on remote QA and PRD API and database servers for integration with MTP.

v10-dge branch contains the latest version of locally developed API and database server, which is under review at https://github.com/PediatricOpenTargets/OpenPedCan-api/pull/65, but v10-dge branch is not deployed remotely.

Therefore, the branch for v11 data release should be developed based on v10-dge. After the v11 update PR is created, previous PRs can be reviewed, to separate all v11 update PR changes into multiple PRs, which could facilitate the reviewing process.

After v11 update PR is approved and merged, the changes will be deployed to QA API HTTP server, and QA API database needs to be updated with v11 database. Then, the updated QA API site needs to be evaluated by FNL team for integration with MTP.

cc @jharenza @chinwallaa @taylordm @afarrel @ewafula @kelseykeith

[logstar]

Updates/progress on this issue:

• Updated OpenPedCan-analysis submodule to v11 release.
• Added 16GB swap space on the EC2 instance that I used for developing OpenPedCan-api, with help from @blackdenc . The swap space prevents docker container from being terminated when exceeding RAM limit.
• Updated db/build_tools/tpm_data_lists.R to be compatible with v11 release. Checked whether v11 data are compatible with db/build_tools/tpm_data_lists.R assertions/assumptions related to cohort, experimental_strategy, Kids_First_Biospecimen_ID, cancer_group, EFO, MONDO, etc.

I will continue working on this issue in the next week, with the following action items:

• Check whether v11 data are compatible with db/build_tools/build_db.R assertions/assumptions.
• Rerun database building procedure.
• Set up local OpenPedCan-api and database instances.
• Check whether v11 data are compatible with src/get_gene_tpm_tbl.R assertions/assumptions.

Let me know if you have questions or suggestions.

[logstar]

Updates/progress on this issue:

• Checked that v11 data are compatible with db/build_tools/build_db.R. Added comments to improve readability.
• Built database using v11 data.
• Set up local OpenPedCan-api and database instances using v11 database.
• Checked that v11 database is compatible with src/get_gene_tpm_tbl.R and src/get_gene_tpm_boxplot_tbl.R.

I will continue working on this issue in the next week, with the following action items:

• Check whether v11 database is compatible with src/get_gene_tpm_boxplot_summary_tbl.R and src/get_gene_tpm_boxplot.R.
• Update and run tests locally for v11 database.
• Review plots and tables generated by the tests.
• Write changelog for this update.
• Load v11 database to dev AWS RDS server.

Let me know if you have questions or suggestions.

[logstar]

I will continue working on the following action items next week:

• Rerun tests locally with pre-built database that is uploaded to s3://d3b-openaccess-us-east-1-prd-pbta/open-targets/api/v0.5.3-beta/.
• Update README.md to clarify that differential gene expression (DGE) database table uses v10 data, whereas TPM database table uses v11 data. As DGE endpoints have not been integrated to MTP, this will not cause inconsistent data versions on MTP.
• Change DEV API instance database configuration to use the v11 database.
• Open a PR for this update. This will update the DEV API instance to use v11 database.
• Rerun tests on DEV API instance.