RTXteam / RTX

Software repo for Team Expander Agent (Oregon State U., Institute for Systems Biology, and Penn State U.)
https://arax.ncats.io/
MIT License
33 stars 20 forks source link

KG2.8.4c Build #2073

Closed ecwood closed 11 months ago

ecwood commented 1 year ago

Hi! KG2.8.4pre is ready to be canonicalized and rolled out.

THE BRANCH FOR THIS ROLLOUT IS: KG2.8.4c THE ARAX-DATABASES.RTX.AI DIRECTORY FOR THIS ROLLOUT IS: /home/rtxconfig/KG2.8.4

1. Build and load KG2c:

@sundareswarpullela

2. Rebuild downstream databases:

The following databases should be rebuilt and copies of them should be put in /home/rtxconfig/KG2.X.Y on arax-databases.rtx.ai. Please use this kind of naming format: mydatabase_v1.0_KG2.X.Y.sqlite.

NOTE: As databases are rebuilt, RTX/code/config_dbs.json will need to be updated to point to their new paths! Push these changes to the branch for this KG2 version, unless the rollout of this KG2 version has already occurred, in which case you should push to master (but first follow the steps described here).

3. Update the ARAX codebase:

All code changes should go in the branch for this KG2 version!

4. Pre-upload databases:

Before rolling out, we need to pre-upload the new databases (referenced in config_dbs.json) to arax.ncats.io and the ITRB SFTP server. These steps can be done well in advance of the rollout; it doesn't hurt anything to do them early.

5. Do the rollout:

6. Final items/clean up:

7. Roll-out to ITRB TEST

8. Roll-out to ITRB PRODUCTION

chunyuma commented 1 year ago

Hi @ecwood, @saramsey, @sundareswarpullela and @amykglen, considering we're going to retire the old DTD in kg2.8.4 rollout by next Wed, we will not need DTD model anymore in the future rollout. I'm wondering if you could help update the following contents in the kg rollout template in the future:

change:

 refreshed DTD @chunyuma
 DTD model @chunyuma (may be skipped - depends on the changes in this KG2 version)
 DTD database @chunyuma (may be skipped - depends on the changes in this KG2 version)
XDTD database @chunyuma

to

refreshed xDTD @chunyuma and @kvnthomas98  
XDTD database @chunyuma and @kvnthomas98  
saramsey commented 1 year ago

We have upgraded: devED, kg2beta, test, beta, and devLM. All verified (via the web UI) to be running against PloverDB/KG2.8.4c via the one-node query graph technique.

The four files that were locally modified in devLM have been stored with git stash.

saramsey commented 1 year ago

All non-skipped pytest tests passing (run in the devED devarea):

rt@d1fd345478a0:/mnt/data/orangeboard/devED/RTX/code/ARAX/test$ pytest -v
============================================ test session starts ============================================
platform linux -- Python 3.9.16, pytest-7.2.2, pluggy-1.0.0 -- /mnt/data/python/Python-3.9.16/bin/python3.9
cachedir: .pytest_cache
rootdir: /mnt/data/orangeboard/devED/RTX/code/ARAX/test
plugins: cov-4.1.0
collected 264 items

test_ARAX_connect.py::test_connect_acetaminophen_to_ptgs1 PASSED                                      [  0%]
test_ARAX_connect.py::test_connect_nodes_1_hop SKIPPED (need --runslow option to run)                 [  0%]
test_ARAX_connect.py::test_connect_nodes_2_hop SKIPPED (need --runslow option to run)                 [  1%]
test_ARAX_connect.py::test_connect_nodes_subgraph_2_hop SKIPPED (need --runslow option to run)        [  1%]
test_ARAX_connect.py::test_connect_nodes_3_disconnected_nodes SKIPPED (need --runslow option to run)  [  1%]
test_ARAX_connect.py::test_1881 SKIPPED (need --runslow option to run)                                [  2%]
test_ARAX_connect.py::test_none_object SKIPPED (need --runslow option to run)                         [  2%]
test_ARAX_expand.py::test_720_multiple_qg_ids_in_different_results SKIPPED (need --runslow option...) [  3%]
test_ARAX_expand.py::test_bte_query SKIPPED (need --runexternal option to run)                        [  3%]
test_ARAX_expand.py::test_single_node_query_with_synonyms PASSED                                      [  3%]
test_ARAX_expand.py::test_single_node_query_with_no_results PASSED                                    [  4%]
test_ARAX_expand.py::test_single_node_query_with_list PASSED                                          [  4%]
test_ARAX_expand.py::test_branched_query SKIPPED (need --runslow option to run)                       [  4%]
test_ARAX_expand.py::test_query_that_expands_same_edge_twice SKIPPED (need --runslow option to run)   [  5%]
test_ARAX_expand.py::test_771_continue_if_no_results_query PASSED                                     [  5%]
test_ARAX_expand.py::test_774_continue_if_no_results_query SKIPPED (need --runslow option to run)     [  6%]
test_ARAX_expand.py::test_curie_list_query PASSED                                                     [  6%]
test_ARAX_expand.py::test_query_with_curies_on_both_ends SKIPPED (need --runslow option to run)       [  6%]
test_ARAX_expand.py::test_query_with_intermediate_curie_node SKIPPED (need --runslow option to run)   [  7%]
test_ARAX_expand.py::test_847_dont_expand_curie_less_edge PASSED                                      [  7%]
test_ARAX_expand.py::test_deduplication_and_self_edges SKIPPED (need --runslow option to run)         [  7%]
test_ARAX_expand.py::test_873_consider_both_gene_and_protein SKIPPED (need --runslow option to run)   [  8%]
test_ARAX_expand.py::test_987_override_node_categories PASSED                                         [  8%]
test_ARAX_expand.py::test_cohd_expand SKIPPED (need --runexternal option to run)                      [  9%]
test_ARAX_expand.py::test_dtd_expand_1 SKIPPED (retire DTD)                                           [  9%]
test_ARAX_expand.py::test_dtd_expand_2 SKIPPED (retire DTD)                                           [  9%]
test_ARAX_expand.py::test_ngd_expand SKIPPED (unconditional skip)                                     [ 10%]
test_ARAX_expand.py::test_chp_expand_1 SKIPPED (need --runexternal option to run)                     [ 10%]
test_ARAX_expand.py::test_chp_expand_2 SKIPPED (need --runexternal option to run)                     [ 10%]
test_ARAX_expand.py::test_genetics_kp SKIPPED (need --runexternal option to run)                      [ 11%]
test_ARAX_expand.py::test_molepro_query SKIPPED (need --runexternal option to run)                    [ 11%]
test_ARAX_expand.py::test_spoke_query SKIPPED (need --runexternal option to run)                      [ 12%]
test_ARAX_expand.py::test_spoke_query_2 SKIPPED (need --runexternal option to run)                    [ 12%]
test_ARAX_expand.py::test_exclude_edge_parallel SKIPPED (need --runslow option to run)                [ 12%]
test_ARAX_expand.py::test_exclude_edge_perpendicular SKIPPED (need --runslow option to run)           [ 13%]
test_ARAX_expand.py::test_exclude_edge_ordering SKIPPED (need --runslow option to run)                [ 13%]
test_ARAX_expand.py::test_exclude_edge_no_results PASSED                                              [ 14%]
test_ARAX_expand.py::test_option_group_query_one_hop PASSED                                           [ 14%]
test_ARAX_expand.py::test_option_group_query_no_results SKIPPED (need --runslow option to run)        [ 14%]
test_ARAX_expand.py::test_category_and_predicate_format PASSED                                        [ 15%]
test_ARAX_expand.py::test_issue_1212 PASSED                                                           [ 15%]
test_ARAX_expand.py::test_issue_1314 PASSED                                                           [ 15%]
test_ARAX_expand.py::test_issue_1236_a SKIPPED (need --runexternal option to run)                     [ 16%]
test_ARAX_expand.py::test_issue_1236_b PASSED                                                         [ 16%]
test_ARAX_expand.py::test_kg2_predicate_hierarchy_reasoning PASSED                                    [ 17%]
test_ARAX_expand.py::test_domain_range_exclusion SKIPPED (Dev testing for domain range exclusion)     [ 17%]
test_ARAX_expand.py::test_issue_1373_pinned_curies SKIPPED (need --runslow option to run)             [ 17%]
test_ARAX_expand.py::test_multiomics_clinical_risk_kp SKIPPED (need --runexternal option to run)      [ 18%]
test_ARAX_expand.py::test_multiomics_wellness_kp SKIPPED (need --runexternal option to run)           [ 18%]
test_ARAX_expand.py::test_multiomics_drug_response_kp SKIPPED (need --runexternal option to run)      [ 18%]
test_ARAX_expand.py::test_multiomics_tumor_gene_mutation_kp SKIPPED (need --runexternal option to...) [ 19%]
test_ARAX_expand.py::test_many_kp_query PASSED                                                        [ 19%]
test_ARAX_expand.py::test_qualified_regulates_query PASSED                                            [ 20%]
test_ARAX_expand.py::test_1516_single_quotes_in_ids PASSED                                            [ 20%]
test_ARAX_expand.py::test_input_curie_remapping PASSED                                                [ 20%]
test_ARAX_expand.py::test_constraint_validation PASSED                                                [ 21%]
test_ARAX_expand.py::test_edge_constraints PASSED                                                     [ 21%]
test_ARAX_expand.py::test_canonical_predicates PASSED                                                 [ 21%]
test_ARAX_expand.py::test_curie_prefix_conversion_1537 SKIPPED (need --runslow option to run)         [ 22%]
test_ARAX_expand.py::test_merging_node_attributes_1450 SKIPPED (need --runslow option to run)         [ 22%]
test_ARAX_expand.py::test_icees_dili SKIPPED (need --runexternal option to run)                       [ 23%]
test_ARAX_expand.py::test_icees_asthma SKIPPED (need --runexternal option to run)                     [ 23%]
test_ARAX_expand.py::test_almost_cycle_1565 SKIPPED (need --runslow option to run)                    [ 23%]
test_ARAX_expand.py::test_fda_approved_query_simple SKIPPED (need --runslow option to run)            [ 24%]
test_ARAX_expand.py::test_fda_approved_query_workflow_a9_egfr_advanced SKIPPED (need --runslow op...) [ 24%]
test_ARAX_expand.py::test_inverted_treats_handling PASSED                                             [ 25%]
test_ARAX_expand.py::test_xdtd_expand PASSED                                                          [ 25%]
test_ARAX_expand.py::test_xdtd_different_categories SKIPPED (need --runslow option to run)            [ 25%]
test_ARAX_expand.py::test_xdtd_multiple_categories PASSED                                             [ 26%]
test_ARAX_expand.py::test_xdtd_different_predicates PASSED                                            [ 26%]
test_ARAX_expand.py::test_xdtd_no_curies PASSED                                                       [ 26%]
test_ARAX_expand.py::test_xdtd_with_other_edges SKIPPED (unconditional skip)                          [ 27%]
test_ARAX_expand.py::test_xdtd_curie_not_in_db PASSED                                                 [ 27%]
test_ARAX_expand.py::test_query_ids_mappings SKIPPED (need --runslow option to run)                   [ 28%]
test_ARAX_expand.py::test_no_query_ids_issue SKIPPED (need --runexternal option to run)               [ 28%]
test_ARAX_expand.py::test_subclass_answers_for_non_pinned_qnodes SKIPPED (need --runslow option t...) [ 28%]
test_ARAX_expand.py::test_kp_list PASSED                                                              [ 29%]
test_ARAX_expand.py::test_missing_epc_attributes PASSED                                               [ 29%]
test_ARAX_filter_kg.py::test_command_definitions PASSED                                               [ 29%]
test_ARAX_filter_kg.py::test_warnings PASSED                                                          [ 30%]
test_ARAX_filter_kg.py::test_error PASSED                                                             [ 30%]
test_ARAX_filter_kg.py::test_edge_key_removal PASSED                                                  [ 31%]
test_ARAX_filter_kg.py::test_default_std_dev SKIPPED (need --runslow option to run)                   [ 31%]
test_ARAX_filter_kg.py::test_std_dev SKIPPED (need --runslow option to run)                           [ 31%]
test_ARAX_filter_kg.py::test_default_top_n SKIPPED (need --runslow option to run)                     [ 32%]
test_ARAX_filter_kg.py::test_remove_property_known_attributes PASSED                                  [ 32%]
test_ARAX_filter_kg.py::test_remove_attribute_known_attributes SKIPPED (need --runslow option to run) [ 32%]
test_ARAX_filter_kg.py::test_provided_by_filter SKIPPED (need --runslow option to run)                [ 33%]
test_ARAX_filter_kg.py::test_stats_error_int_threshold SKIPPED (need --runslow option to run)         [ 33%]
test_ARAX_filter_kg.py::test_tuple_bug PASSED                                                         [ 34%]
test_ARAX_filter_results.py::test_command_definitions PASSED                                          [ 34%]
test_ARAX_filter_results.py::test_n_results PASSED                                                    [ 34%]
test_ARAX_filter_results.py::test_no_results PASSED                                                   [ 35%]
test_ARAX_filter_results.py::test_prune SKIPPED (need --runslow option to run)                        [ 35%]
test_ARAX_filter_results.py::test_warning PASSED                                                      [ 35%]
test_ARAX_filter_results.py::test_sort_by_edge_attribute SKIPPED (need --runslow option to run)       [ 36%]
test_ARAX_filter_results.py::test_sort_by_node_attribute PASSED                                       [ 36%]
test_ARAX_filter_results.py::test_sort_by_score PASSED                                                [ 37%]
test_ARAX_filter_results.py::test_issue1506 SKIPPED (need --runexternal option to run)                [ 37%]
test_ARAX_infer.py::test_xdtd_infer_castleman_disease_1 PASSED                                        [ 37%]
test_ARAX_infer.py::test_xdtd_infer_castleman_disease_2 PASSED                                        [ 38%]
test_ARAX_infer.py::test_xdtd_with_qg PASSED                                                          [ 38%]
test_ARAX_infer.py::test_xdtd_with_qg2 PASSED                                                         [ 39%]
test_ARAX_infer.py::test_xdtd_with_qg3 PASSED                                                         [ 39%]
test_ARAX_infer.py::test_xdtd_with_only_qg PASSED                                                     [ 39%]
test_ARAX_infer.py::test_xcrg_infer_bomeol SKIPPED (need --runslow option to run)                     [ 40%]
test_ARAX_infer.py::test_xcrg_with_qg1 SKIPPED (need --runslow option to run)                         [ 40%]
test_ARAX_infer.py::test_xcrg_with_qg2 SKIPPED (need --runslow option to run)                         [ 40%]
test_ARAX_infer.py::test_xcrg_with_only_qg SKIPPED (need --runslow option to run)                     [ 41%]
test_ARAX_infer.py::test_xcrg_infer_dsl SKIPPED (need --runslow option to run)                        [ 41%]
test_ARAX_json_queries.py::test_query_by_query_graph_2 PASSED                                         [ 42%]
test_ARAX_json_queries.py::test_ngd_added PASSED                                                      [ 42%]
test_ARAX_json_queries.py::test_drug_disease_query SKIPPED (need --runslow option to run)             [ 42%]
test_ARAX_json_queries.py::test_workflow1 PASSED                                                      [ 43%]
test_ARAX_json_queries.py::test_workflow2 SKIPPED (need --runslow option to run)                      [ 43%]
test_ARAX_messenger.py::test_create_message_basic PASSED                                              [ 43%]
test_ARAX_messenger.py::test_create_message_node_edge_types PASSED                                    [ 44%]
test_ARAX_messenger.py::test_add_qnode_basic PASSED                                                   [ 44%]
test_ARAX_messenger.py::test_add_qnode_curie_scalar PASSED                                            [ 45%]
test_ARAX_messenger.py::test_add_qnode_curie_list PASSED                                              [ 45%]
test_ARAX_messenger.py::test_add_qnode_name PASSED                                                    [ 45%]
test_ARAX_messenger.py::test_add_qnode_type PASSED                                                    [ 46%]
test_ARAX_messenger.py::test_add_qnode_group_id_is_set_false PASSED                                   [ 46%]
test_ARAX_messenger.py::test_add_qnode_bad_name PASSED                                                [ 46%]
test_ARAX_messenger.py::test_add_qnode_duplicate_key PASSED                                           [ 47%]
test_ARAX_messenger.py::test_add_qedge_duplicate_key PASSED                                           [ 47%]
test_ARAX_messenger.py::test_add_qnode_bad_parameters PASSED                                          [ 48%]
test_ARAX_messenger.py::test_add_qedge_multitest PASSED                                               [ 48%]
test_ARAX_overlay.py::test_jaccard PASSED                                                             [ 48%]
test_ARAX_overlay.py::test_add_node_pmids PASSED                                                      [ 49%]
test_ARAX_overlay.py::test_compute_ngd_virtual PASSED                                                 [ 49%]
test_ARAX_overlay.py::test_compute_ngd_attribute PASSED                                               [ 50%]
test_ARAX_overlay.py::test_FET_ex1 PASSED                                                             [ 50%]
test_ARAX_overlay.py::test_FET_ex2 SKIPPED (need --runslow option to run)                             [ 50%]
test_ARAX_overlay.py::test_paired_concept_frequency_virtual SKIPPED (need --runslow option to run)    [ 51%]
test_ARAX_overlay.py::test_paired_concept_frequency_attribute SKIPPED (need --runslow option to run)  [ 51%]
test_ARAX_overlay.py::test_observed_expected_ratio_virtual SKIPPED (need --runslow option to run)     [ 51%]
test_ARAX_overlay.py::test_observed_expected_ratio_attribute SKIPPED (need --runslow option to run)   [ 52%]
test_ARAX_overlay.py::test_chi_square_virtual SKIPPED (need --runslow option to run)                  [ 52%]
test_ARAX_overlay.py::test_chi_square_attribute SKIPPED (need --runslow option to run)                [ 53%]
test_ARAX_overlay.py::test_predict_drug_treats_disease_virtual SKIPPED (retire DTD)                   [ 53%]
test_ARAX_overlay.py::test_predict_drug_treats_disease_attribute SKIPPED (retire DTD)                 [ 53%]
test_ARAX_overlay.py::test_issue_832 SKIPPED (retire DTD)                                             [ 54%]
test_ARAX_overlay.py::test_issue_832_non_drug SKIPPED (retire DTD)                                    [ 54%]
test_ARAX_overlay.py::test_issue_840 SKIPPED (need --runslow option to run)                           [ 54%]
test_ARAX_overlay.py::test_issue_840_non_drug SKIPPED (need --runslow option to run)                  [ 55%]
test_ARAX_overlay.py::test_issue_892 SKIPPED (retire DTD)                                             [ 55%]
test_ARAX_overlay.py::test_overlay_exposures_data_virtual SKIPPED (need --runexternal option to run)  [ 56%]
test_ARAX_overlay.py::test_overlay_exposures_data_attribute SKIPPED (need --runexternal option to...) [ 56%]
test_ARAX_overlay.py::test_overlay_clinical_info_no_ids SKIPPED (need --runslow option to run)        [ 56%]
test_ARAX_overlay.py::test_missing_ngd_pmids SKIPPED (need --runslow option to run)                   [ 57%]
test_ARAX_overlay.py::test_jaccard_not_above_1 SKIPPED (need --runslow option to run)                 [ 57%]
test_ARAX_overlay.py::test_ngd_sqlite_syntax_error SKIPPED (need --runslow option to run)             [ 57%]
test_ARAX_query.py::test_query_by_query_graph_2 PASSED                                                [ 58%]
test_ARAX_resultify.py::test01 PASSED                                                                 [ 58%]
test_ARAX_resultify.py::test02 PASSED                                                                 [ 59%]
test_ARAX_resultify.py::test03 PASSED                                                                 [ 59%]
test_ARAX_resultify.py::test04 PASSED                                                                 [ 59%]
test_ARAX_resultify.py::test05 PASSED                                                                 [ 60%]
test_ARAX_resultify.py::test07 PASSED                                                                 [ 60%]
test_ARAX_resultify.py::test08 PASSED                                                                 [ 60%]
test_ARAX_resultify.py::test09 SKIPPED (need --runslow option to run)                                 [ 61%]
test_ARAX_resultify.py::test10 PASSED                                                                 [ 61%]
test_ARAX_resultify.py::test_example1 SKIPPED (need --runslow option to run)                          [ 62%]
test_ARAX_resultify.py::test_bfs PASSED                                                               [ 62%]
test_ARAX_resultify.py::test_bfs_in_essence_code PASSED                                               [ 62%]
test_ARAX_resultify.py::test_issue680 SKIPPED (need --runslow option to run)                          [ 63%]
test_ARAX_resultify.py::test_issue686a PASSED                                                         [ 63%]
test_ARAX_resultify.py::test_issue686b PASSED                                                         [ 64%]
test_ARAX_resultify.py::test_issue686c PASSED                                                         [ 64%]
test_ARAX_resultify.py::test_issue687 PASSED                                                          [ 64%]
test_ARAX_resultify.py::test_issue727 PASSED                                                          [ 65%]
test_ARAX_resultify.py::test_issue731 PASSED                                                          [ 65%]
test_ARAX_resultify.py::test_issue731b SKIPPED (need --runslow option to run)                         [ 65%]
test_ARAX_resultify.py::test_issue731c PASSED                                                         [ 66%]
test_ARAX_resultify.py::test_issue740 PASSED                                                          [ 66%]
test_ARAX_resultify.py::test_issue692 PASSED                                                          [ 67%]
test_ARAX_resultify.py::test_issue692b PASSED                                                         [ 67%]
test_ARAX_resultify.py::test_issue720_1 SKIPPED (need --runslow option to run)                        [ 67%]
test_ARAX_resultify.py::test_issue720_2 SKIPPED (need --runslow option to run)                        [ 68%]
test_ARAX_resultify.py::test_issue720_3 SKIPPED (need --runslow option to run)                        [ 68%]
test_ARAX_resultify.py::test_issue833_extraneous_intermediate_nodes PASSED                            [ 68%]
test_ARAX_resultify.py::test_single_node PASSED                                                       [ 69%]
test_ARAX_resultify.py::test_parallel_edges_between_nodes PASSED                                      [ 69%]
test_ARAX_resultify.py::test_issue912_clean_up_kg PASSED                                              [ 70%]
test_ARAX_resultify.py::test_issue1119_a SKIPPED (need --runslow option to run)                       [ 70%]
test_ARAX_resultify.py::test_issue1119_b SKIPPED (need --runslow option to run)                       [ 70%]
test_ARAX_resultify.py::test_issue1119_c SKIPPED (need --runslow option to run)                       [ 71%]
test_ARAX_resultify.py::test_issue1119_d SKIPPED (need --runslow option to run)                       [ 71%]
test_ARAX_resultify.py::test_issue1146_a SKIPPED (need --runslow option to run)                       [ 71%]
test_ARAX_resultify.py::test_disconnected_qg PASSED                                                   [ 72%]
test_ARAX_resultify.py::test_recompute_qg_keys PASSED                                                 [ 72%]
test_ARAX_resultify.py::test_multi_node_edgeless_qg PASSED                                            [ 73%]
test_ARAX_resultify.py::test_issue_1446 SKIPPED (need --runslow option to run)                        [ 73%]
test_ARAX_resultify.py::test_issue_1848 SKIPPED (need --runslow option to run)                        [ 73%]
test_ARAX_resultify.py::test_node_binding_query_id_one_hop_single_input_curie PASSED                  [ 74%]
test_ARAX_resultify.py::test_node_binding_query_id_one_hop_multiple_input_curies PASSED               [ 74%]
test_ARAX_resultify.py::test_node_binding_query_id_two_hop_double_pinned SKIPPED (need --runslow ...) [ 75%]
test_ARAX_resultify.py::test_missing_chp_results SKIPPED (need --runexternal option to run)           [ 75%]
test_ARAX_resultify.py::test_too_few_results SKIPPED (need --runslow option to run)                   [ 75%]
test_ARAX_resultify.py::test_issue1923_multiple_essence_candidates_subclass SKIPPED (need --runsl...) [ 76%]
test_ARAX_standup_queries.py::test_gene_to_pathway_issue_9 PASSED                                     [ 76%]
test_ARAX_standup_queries.py::test_chemicals_to_gene_issue_10 PASSED                                  [ 76%]
test_ARAX_standup_queries.py::test_named_thing_associated_with_acrocynaosis_issue_12 PASSED           [ 77%]
test_ARAX_standup_queries.py::test_chemical_substances_correlated_with_asthma_issue_18 SKIPPED (n...) [ 77%]
test_ARAX_standup_queries.py::test_diseases_treated_by_drug_issue_20 SKIPPED (need --runslow opti...) [ 78%]
test_ARAX_standup_queries.py::test_chemical_substances_that_down_regulate_STK11_issue_28 SKIPPED      [ 78%]
test_ARAX_standup_queries.py::test_phenotypes_for_angel_shaped_phalango_epiphyseal_dysplasia_issue_33 SKIPPED [ 78%]
test_ARAX_synonymizer.py::test_example_6b PASSED                                                      [ 79%]
test_ARAX_synonymizer.py::test_example_9 PASSED                                                       [ 79%]
test_ARAX_synonymizer.py::test_example_10 PASSED                                                      [ 79%]
test_ARAX_synonymizer.py::test_example_11 PASSED                                                      [ 80%]
test_ARAX_synonymizer.py::test_example_12 PASSED                                                      [ 80%]
test_ARAX_synonymizer.py::test_get_canonical_curies_simple PASSED                                     [ 81%]
test_ARAX_synonymizer.py::test_get_canonical_curies_single_curie PASSED                               [ 81%]
test_ARAX_synonymizer.py::test_get_canonical_curies_unrecognized PASSED                               [ 81%]
test_ARAX_synonymizer.py::test_get_canonical_curies_by_names PASSED                                   [ 82%]
test_ARAX_synonymizer.py::test_get_canonical_curies_single_name PASSED                                [ 82%]
test_ARAX_synonymizer.py::test_get_canonical_curies_by_names_and_curies PASSED                        [ 82%]
test_ARAX_synonymizer.py::test_get_canonical_curies_return_all_categories PASSED                      [ 83%]
test_ARAX_synonymizer.py::test_get_equivalent_nodes PASSED                                            [ 83%]
test_ARAX_synonymizer.py::test_get_equivalent_nodes_by_name PASSED                                    [ 84%]
test_ARAX_synonymizer.py::test_get_equivalent_nodes_by_curies_and_names PASSED                        [ 84%]
test_ARAX_synonymizer.py::test_get_normalizer_results PASSED                                          [ 84%]
test_ARAX_synonymizer.py::test_improper_curie_prefix_capitalization PASSED                            [ 85%]
test_ARAX_synonymizer.py::test_approximate_name_based_matching PASSED                                 [ 85%]
test_ARAX_synonymizer.py::test_entity_controller_input_no_format PASSED                               [ 85%]
test_ARAX_synonymizer.py::test_entity_controller_input_minimal_format PASSED                          [ 86%]
test_ARAX_synonymizer.py::test_cluster_graphs PASSED                                                  [ 86%]
test_ARAX_translate.py::test_lookup PASSED                                                            [ 87%]
test_ARAX_translate.py::test_fill_success PASSED                                                      [ 87%]
test_ARAX_translate.py::test_fill_error PASSED                                                        [ 87%]
test_ARAX_translate.py::test_score PASSED                                                             [ 88%]
test_ARAX_translate.py::test_bind PASSED                                                              [ 88%]
test_ARAX_translate.py::test_complete_results PASSED                                                  [ 89%]
test_ARAX_translate.py::test_filter_results_top_n PASSED                                              [ 89%]
test_ARAX_translate.py::test_overlay_after_lookup PASSED                                              [ 89%]
test_ARAX_translate.py::test_connect_knodes_2_nodes SKIPPED (need --runslow option to run)            [ 90%]
test_ARAX_translate.py::test_connect_knodes_3_nodes SKIPPED (need --runslow option to run)            [ 90%]
test_ARAX_translate.py::test_unknown_operation PASSED                                                 [ 90%]
test_ARAX_workflows.py::test_option_group_id PASSED                                                   [ 91%]
test_ARAX_workflows.py::test_exclude PASSED                                                           [ 91%]
test_ARAX_workflows.py::test_example_2 SKIPPED (need --runslow option to run)                         [ 92%]
test_ARAX_workflows.py::test_example_3 SKIPPED (need --runslow option to run)                         [ 92%]
test_ARAX_workflows.py::test_FET_example_1 SKIPPED (need --runslow option to run)                     [ 92%]
test_ARAX_workflows.py::test_FET_example_2 SKIPPED (FW: This now runs increadibly slowly and uses...) [ 93%]
test_ARAX_workflows.py::test_FET_example_3 SKIPPED (need issue#846 to be solved)                      [ 93%]
test_ARAX_workflows.py::test_FET_example_4 SKIPPED (need --runslow option to run)                     [ 93%]
test_ARAX_workflows.py::test_FET_ranking_1 SKIPPED (need --runslow option to run)                     [ 94%]
test_ARAX_workflows.py::test_example_2_kg2 SKIPPED (need --runslow option to run)                     [ 94%]
test_ARAX_workflows.py::test_clinical_overlay_example1 SKIPPED (need --runslow option to run)         [ 95%]
test_ARAX_workflows.py::test_clinical_overlay_example2 SKIPPED (redundant if the test_clinical_ov...) [ 95%]
test_ARAX_workflows.py::test_two_hop_based_on_types_1 SKIPPED (redundant if test_one_hop_based_on...) [ 95%]
test_ARAX_workflows.py::test_one_hop_based_on_types_1 SKIPPED (need --runslow option to run)          [ 96%]
test_ARAX_workflows.py::test_one_hop_kitchen_sink_BTE_1 SKIPPED (Work in progress (and takes a ve...) [ 96%]
test_ARAX_workflows.py::test_one_hop_kitchen_sink_BTE_2 SKIPPED (Work in progress (and takes a ve...) [ 96%]
test_ARAX_workflows.py::test_FET_ranking_2 PASSED                                                     [ 97%]
test_ARAX_workflows.py::test_genetics_kp_ranking SKIPPED (need --runexternal option to run)           [ 97%]
test_ARAX_workflows.py::test_ranker_float_error_ex1 SKIPPED (need --runslow option to run)            [ 98%]
test_ARAX_workflows.py::test_ranker_float_error_ex2 SKIPPED (need --runexternal option to run)        [ 98%]
test_ARAX_workflows.py::test_cmap_ranking SKIPPED (need --runexternal option to run)                  [ 98%]
test_ARAX_workflows.py::test_ranker_float_error_ex3 SKIPPED (need --runslow option to run)            [ 99%]
test_ARAX_workflows.py::test_issue_1848 SKIPPED (need --runexternal option to run)                    [ 99%]
test_ARAX_workflows.py::test_example_3_issue_679 SKIPPED (need --runslow option to run)               [100%]

============================================= warnings summary ==============================================
saramsey commented 1 year ago

Shutting down the KG2.8.3c Neo4j instance, per the checklist. Confirming with DNS that this is in fact kg2canonicalized.rtx.ai:

10-197-139-17:~ sramsey$ nslookup kg2-8-3c.rtx.ai
Server:     128.193.15.12
Address:    128.193.15.12#53

Non-authoritative answer:
kg2-8-3c.rtx.ai canonical name = kg2canonicalized.rtx.ai.
Name:   kg2canonicalized.rtx.ai
Address: 52.40.12.47
saramsey commented 1 year ago

Confirming that kg2cplover.rtx.ai is running KG2.8.3c:

(venv) 10-197-139-17:PloverDB sramsey$ pytest -vs test/test.py::test_version --endpoint http://kg2cplover.rtx.ai:9990
============================================================ test session starts =============================================================
platform darwin -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0 -- /opt/homebrew/opt/python@3.11/bin/python3.11
cachedir: .pytest_cache
rootdir: /Users/sramsey/Documents/Work/Proj/ploverdb/PloverDB-main/PloverDB
collected 1 item

test/test.py::test_version n00: {'RTX:KG2c': ['RTX-KG2.8.3c', 'biolink:InformationContentEntity', []]}
{'edges': {}, 'nodes': {'n00': {'RTX:KG2c': ['RTX-KG2.8.3c', 'biolink:InformationContentEntity', []]}}}
PASSED

============================================================= 1 passed in 2.54s ==============================================================
saramsey commented 1 year ago

We are skipping this step for now, in case we need to do some Neo4j debugging/troubleshooting for KG2.8.4c:

turn off the new KG2pre version's neo4j instance

saramsey commented 1 year ago

At 10:15 AM PDT we can start testing to see if the new PloverDB is working on kg2cploverdb.ci.transltr.io.

saramsey commented 1 year ago

We are using this DSL command (courtesy of @sundareswarpullela ) to test the SemMedDB exclusion filtering:

add_qnode(ids=UMLS:C1510438, key=n00)
add_qnode(categories=biolink:Disease, key=n01)
add_qedge(subject=n00, object=n01, key=e00, predicates=biolink:diagnoses)
expand(kp=infores:rtx-kg2)
resultify()

and the equivalent JSON:

{
   "edges": {
      "e00": {
         "subject":   "n00",
         "object":    "n01",
         "predicates": ["biolink:diagnoses"]
      }
   },
   "nodes": {
      "n00": {
         "ids":        ["UMLS:C1510438"]
      },
      "n01": {
         "categories":  ["biolink:Disease"]
      }
   }
}
saramsey commented 1 year ago

I am doing the database download into cicd.rtx.ai right now.... we are using a locally hacked version of ARAX_database_manager.py until such time as #2098 is addressed:

Screenshot 2023-08-17 at 12 17 52 PM
saramsey commented 1 year ago

PloverDB/KG2.8.4c is running (with the latest code) in kg2cploverdb.ci.transltr.io:

curl -X GET -H 'accept: application/json' https://kg2cploverdb.ci.transltr.io/code_version
"HEAD: HEAD; Date: 2023-08-17

and passing all tests:

(venv) 10-197-139-17:PloverDB sramsey$ pytest -v test/test.py --endpoint https://kg2cploverdb.ci.transltr.io
============================= test session starts ==============================
platform darwin -- Python 3.9.17, pytest-7.4.0, pluggy-1.2.0 -- /Users/sramsey/Documents/Work/Proj/ploverdb/main/venv/bin/python3.9
cachedir: .pytest_cache
rootdir: /Users/sramsey/Documents/Work/Proj/ploverdb/main/PloverDB
collected 32 items

test/test.py::test_1 PASSED                                              [  3%]
test/test.py::test_2 PASSED                                                      [  6%]
test/test.py::test_3 PASSED                                                      [  9%]
test/test.py::test_4 PASSED                                                      [ 12%]
test/test.py::test_5 PASSED                                                      [ 15%]
test/test.py::test_6 PASSED                                                      [ 18%]
test/test.py::test_7 PASSED                                                      [ 21%]
test/test.py::test_8 PASSED                                                      [ 25%]
test/test.py::test_9 PASSED                                                      [ 28%]
test/test.py::test_10 PASSED                                                     [ 31%]
test/test.py::test_11 PASSED                                                     [ 34%]
test/test.py::test_12 PASSED                                                     [ 37%]
test/test.py::test_13 PASSED                                                     [ 40%]
test/test.py::test_14 PASSED                                                     [ 43%]
test/test.py::test_15 PASSED                                                     [ 46%]
test/test.py::test_16 PASSED                                                     [ 50%]
test/test.py::test_17 PASSED                                                     [ 53%]
test/test.py::test_18 PASSED                                                     [ 56%]
test/test.py::test_19 PASSED                                                     [ 59%]
test/test.py::test_20 PASSED                                                     [ 62%]
test/test.py::test_21 PASSED                                                     [ 65%]
test/test.py::test_22 PASSED                                                     [ 68%]
test/test.py::test_23 PASSED                                                     [ 71%]
test/test.py::test_24 PASSED                                                     [ 75%]
test/test.py::test_25 PASSED                                                     [ 78%]
test/test.py::test_26 PASSED                                                     [ 81%]
test/test.py::test_27 PASSED                                                     [ 84%]
test/test.py::test_28 PASSED                                                     [ 87%]
test/test.py::test_29 PASSED                                                     [ 90%]
test/test.py::test_30 PASSED                                                     [ 93%]
test/test.py::test_31 PASSED                                                     [ 96%]
test/test.py::test_version PASSED                                                [100%]

================================= 32 passed in 35.76s ==================================
saramsey commented 1 year ago

In master, I have updated config_dbs so that under plover, the dev URL points to https://kg2cploverdb.ci.transltr.io.

Screenshot 2023-08-17 at 1 21 37 PM
saramsey commented 1 year ago

Confirmed, the newly deployed arax.ci.transltr.io, in the Expand module, is in fact querying the kg2.ci.transltr.io KP (and also confirmed that via the latter, it is querying PloverDB/KG2.8.4c).

saramsey commented 1 year ago

In the devED area inside the rtx1 container on arax.ncats.io, using current master branch code (prior to commit ba93dbd), all pytest tests are passing:

rt@d1fd345478a0:/mnt/data/orangeboard/devED/RTX/code/ARAX/test$ pytest -v
============================= test session starts ==============================
platform linux -- Python 3.9.16, pytest-7.2.2, pluggy-1.0.0 -- /mnt/data/python/Python-3.9.16/bin/python3.9
cachedir: .pytest_cache
rootdir: /mnt/data/orangeboard/devED/RTX/code/ARAX/test
plugins: cov-4.1.0
collected 264 items

test_ARAX_connect.py::test_connect_acetaminophen_to_ptgs1 PASSED         [  0%]
test_ARAX_connect.py::test_connect_nodes_1_hop SKIPPED (need --runsl...) [  0%]
test_ARAX_connect.py::test_connect_nodes_2_hop SKIPPED (need --runsl...) [  1%]
test_ARAX_connect.py::test_connect_nodes_subgraph_2_hop SKIPPED (nee...) [  1%]
test_ARAX_connect.py::test_connect_nodes_3_disconnected_nodes SKIPPED    [  1%]
test_ARAX_connect.py::test_1881 SKIPPED (need --runslow option to run)   [  2%]
test_ARAX_connect.py::test_none_object SKIPPED (need --runslow optio...) [  2%]
test_ARAX_expand.py::test_720_multiple_qg_ids_in_different_results SKIPPED [  3%]
test_ARAX_expand.py::test_bte_query SKIPPED (need --runexternal opti...) [  3%]
test_ARAX_expand.py::test_single_node_query_with_synonyms PASSED         [  3%]
test_ARAX_expand.py::test_single_node_query_with_no_results PASSED       [  4%]
test_ARAX_expand.py::test_single_node_query_with_list PASSED             [  4%]
test_ARAX_expand.py::test_branched_query SKIPPED (need --runslow opt...) [  4%]
test_ARAX_expand.py::test_query_that_expands_same_edge_twice SKIPPED     [  5%]
test_ARAX_expand.py::test_771_continue_if_no_results_query PASSED        [  5%]
test_ARAX_expand.py::test_774_continue_if_no_results_query SKIPPED (...) [  6%]
test_ARAX_expand.py::test_curie_list_query PASSED                        [  6%]
test_ARAX_expand.py::test_query_with_curies_on_both_ends SKIPPED (ne...) [  6%]
test_ARAX_expand.py::test_query_with_intermediate_curie_node SKIPPED     [  7%]
test_ARAX_expand.py::test_847_dont_expand_curie_less_edge PASSED         [  7%]
test_ARAX_expand.py::test_deduplication_and_self_edges SKIPPED (need...) [  7%]
test_ARAX_expand.py::test_873_consider_both_gene_and_protein SKIPPED     [  8%]
test_ARAX_expand.py::test_987_override_node_categories PASSED            [  8%]
test_ARAX_expand.py::test_cohd_expand SKIPPED (need --runexternal op...) [  9%]
test_ARAX_expand.py::test_dtd_expand_1 SKIPPED (retire DTD)              [  9%]
test_ARAX_expand.py::test_dtd_expand_2 SKIPPED (retire DTD)              [  9%]
test_ARAX_expand.py::test_ngd_expand SKIPPED (unconditional skip)        [ 10%]
test_ARAX_expand.py::test_chp_expand_1 SKIPPED (need --runexternal o...) [ 10%]
test_ARAX_expand.py::test_chp_expand_2 SKIPPED (need --runexternal o...) [ 10%]
test_ARAX_expand.py::test_genetics_kp SKIPPED (need --runexternal op...) [ 11%]
test_ARAX_expand.py::test_molepro_query SKIPPED (need --runexternal ...) [ 11%]
test_ARAX_expand.py::test_spoke_query SKIPPED (need --runexternal op...) [ 12%]
test_ARAX_expand.py::test_spoke_query_2 SKIPPED (need --runexternal ...) [ 12%]
test_ARAX_expand.py::test_exclude_edge_parallel SKIPPED (need --runs...) [ 12%]
test_ARAX_expand.py::test_exclude_edge_perpendicular SKIPPED (need -...) [ 13%]
test_ARAX_expand.py::test_exclude_edge_ordering SKIPPED (need --runs...) [ 13%]
test_ARAX_expand.py::test_exclude_edge_no_results PASSED                 [ 14%]
test_ARAX_expand.py::test_option_group_query_one_hop PASSED              [ 14%]
test_ARAX_expand.py::test_option_group_query_no_results SKIPPED (nee...) [ 14%]
test_ARAX_expand.py::test_category_and_predicate_format PASSED           [ 15%]
test_ARAX_expand.py::test_issue_1212 PASSED                              [ 15%]
test_ARAX_expand.py::test_issue_1314 PASSED                              [ 15%]
test_ARAX_expand.py::test_issue_1236_a SKIPPED (need --runexternal o...) [ 16%]
test_ARAX_expand.py::test_issue_1236_b PASSED                            [ 16%]
test_ARAX_expand.py::test_kg2_predicate_hierarchy_reasoning PASSED       [ 17%]
test_ARAX_expand.py::test_domain_range_exclusion SKIPPED (Dev testin...) [ 17%]
test_ARAX_expand.py::test_issue_1373_pinned_curies SKIPPED (need --r...) [ 17%]
test_ARAX_expand.py::test_multiomics_clinical_risk_kp SKIPPED (need ...) [ 18%]
test_ARAX_expand.py::test_multiomics_wellness_kp SKIPPED (need --run...) [ 18%]
test_ARAX_expand.py::test_multiomics_drug_response_kp SKIPPED (need ...) [ 18%]
test_ARAX_expand.py::test_multiomics_tumor_gene_mutation_kp SKIPPED      [ 19%]
test_ARAX_expand.py::test_many_kp_query PASSED                           [ 19%]
test_ARAX_expand.py::test_qualified_regulates_query PASSED               [ 20%]
test_ARAX_expand.py::test_1516_single_quotes_in_ids PASSED               [ 20%]
test_ARAX_expand.py::test_input_curie_remapping PASSED                   [ 20%]
test_ARAX_expand.py::test_constraint_validation PASSED                   [ 21%]
test_ARAX_expand.py::test_edge_constraints PASSED                        [ 21%]
test_ARAX_expand.py::test_canonical_predicates PASSED                    [ 21%]
test_ARAX_expand.py::test_curie_prefix_conversion_1537 SKIPPED (need...) [ 22%]
test_ARAX_expand.py::test_merging_node_attributes_1450 SKIPPED (need...) [ 22%]
test_ARAX_expand.py::test_icees_dili SKIPPED (need --runexternal opt...) [ 23%]
test_ARAX_expand.py::test_icees_asthma SKIPPED (need --runexternal o...) [ 23%]
test_ARAX_expand.py::test_almost_cycle_1565 SKIPPED (need --runslow ...) [ 23%]
test_ARAX_expand.py::test_fda_approved_query_simple SKIPPED (need --...) [ 24%]
test_ARAX_expand.py::test_fda_approved_query_workflow_a9_egfr_advanced SKIPPED [ 24%]
test_ARAX_expand.py::test_inverted_treats_handling PASSED                [ 25%]
test_ARAX_expand.py::test_xdtd_expand PASSED                             [ 25%]
test_ARAX_expand.py::test_xdtd_different_categories SKIPPED (need --...) [ 25%]
test_ARAX_expand.py::test_xdtd_multiple_categories PASSED                [ 26%]
test_ARAX_expand.py::test_xdtd_different_predicates PASSED               [ 26%]
test_ARAX_expand.py::test_xdtd_no_curies PASSED                          [ 26%]
test_ARAX_expand.py::test_xdtd_with_other_edges SKIPPED (uncondition...) [ 27%]
test_ARAX_expand.py::test_xdtd_curie_not_in_db PASSED                    [ 27%]
test_ARAX_expand.py::test_query_ids_mappings SKIPPED (need --runslow...) [ 28%]
test_ARAX_expand.py::test_no_query_ids_issue SKIPPED (need --runexte...) [ 28%]
test_ARAX_expand.py::test_subclass_answers_for_non_pinned_qnodes SKIPPED [ 28%]
test_ARAX_expand.py::test_kp_list PASSED                                 [ 29%]
test_ARAX_expand.py::test_missing_epc_attributes PASSED                  [ 29%]
test_ARAX_filter_kg.py::test_command_definitions PASSED                  [ 29%]
test_ARAX_filter_kg.py::test_warnings PASSED                             [ 30%]
test_ARAX_filter_kg.py::test_error PASSED                                [ 30%]
test_ARAX_filter_kg.py::test_edge_key_removal PASSED                     [ 31%]
test_ARAX_filter_kg.py::test_default_std_dev SKIPPED (need --runslow...) [ 31%]
test_ARAX_filter_kg.py::test_std_dev SKIPPED (need --runslow option ...) [ 31%]
test_ARAX_filter_kg.py::test_default_top_n SKIPPED (need --runslow o...) [ 32%]
test_ARAX_filter_kg.py::test_remove_property_known_attributes PASSED     [ 32%]
test_ARAX_filter_kg.py::test_remove_attribute_known_attributes SKIPPED   [ 32%]
test_ARAX_filter_kg.py::test_provided_by_filter SKIPPED (need --runs...) [ 33%]
test_ARAX_filter_kg.py::test_stats_error_int_threshold SKIPPED (need...) [ 33%]
test_ARAX_filter_kg.py::test_tuple_bug PASSED                            [ 34%]
test_ARAX_filter_results.py::test_command_definitions PASSED             [ 34%]
test_ARAX_filter_results.py::test_n_results PASSED                       [ 34%]
test_ARAX_filter_results.py::test_no_results PASSED                      [ 35%]
test_ARAX_filter_results.py::test_prune SKIPPED (need --runslow opti...) [ 35%]
test_ARAX_filter_results.py::test_warning PASSED                         [ 35%]
test_ARAX_filter_results.py::test_sort_by_edge_attribute SKIPPED (ne...) [ 36%]
test_ARAX_filter_results.py::test_sort_by_node_attribute PASSED          [ 36%]
test_ARAX_filter_results.py::test_sort_by_score PASSED                   [ 37%]
test_ARAX_filter_results.py::test_issue1506 SKIPPED (need --runexter...) [ 37%]
test_ARAX_infer.py::test_xdtd_infer_castleman_disease_1 PASSED           [ 37%]
test_ARAX_infer.py::test_xdtd_infer_castleman_disease_2 PASSED           [ 38%]
test_ARAX_infer.py::test_xdtd_with_qg PASSED                             [ 38%]
test_ARAX_infer.py::test_xdtd_with_qg2 PASSED                            [ 39%]
test_ARAX_infer.py::test_xdtd_with_qg3 PASSED                            [ 39%]
test_ARAX_infer.py::test_xdtd_with_only_qg PASSED                        [ 39%]
test_ARAX_infer.py::test_xcrg_infer_bomeol SKIPPED (need --runslow o...) [ 40%]
test_ARAX_infer.py::test_xcrg_with_qg1 SKIPPED (need --runslow optio...) [ 40%]
test_ARAX_infer.py::test_xcrg_with_qg2 SKIPPED (need --runslow optio...) [ 40%]
test_ARAX_infer.py::test_xcrg_with_only_qg SKIPPED (need --runslow o...) [ 41%]
test_ARAX_infer.py::test_xcrg_infer_dsl SKIPPED (need --runslow opti...) [ 41%]
test_ARAX_json_queries.py::test_query_by_query_graph_2 PASSED            [ 42%]
test_ARAX_json_queries.py::test_ngd_added PASSED                         [ 42%]
test_ARAX_json_queries.py::test_drug_disease_query SKIPPED (need --r...) [ 42%]
test_ARAX_json_queries.py::test_workflow1 PASSED                         [ 43%]
test_ARAX_json_queries.py::test_workflow2 SKIPPED (need --runslow op...) [ 43%]
test_ARAX_messenger.py::test_create_message_basic PASSED                 [ 43%]
test_ARAX_messenger.py::test_create_message_node_edge_types PASSED       [ 44%]
test_ARAX_messenger.py::test_add_qnode_basic PASSED                      [ 44%]
test_ARAX_messenger.py::test_add_qnode_curie_scalar PASSED               [ 45%]
test_ARAX_messenger.py::test_add_qnode_curie_list PASSED                 [ 45%]
test_ARAX_messenger.py::test_add_qnode_name PASSED                       [ 45%]
test_ARAX_messenger.py::test_add_qnode_type PASSED                       [ 46%]
test_ARAX_messenger.py::test_add_qnode_group_id_is_set_false PASSED      [ 46%]
test_ARAX_messenger.py::test_add_qnode_bad_name PASSED                   [ 46%]
test_ARAX_messenger.py::test_add_qnode_duplicate_key PASSED              [ 47%]
test_ARAX_messenger.py::test_add_qedge_duplicate_key PASSED              [ 47%]
test_ARAX_messenger.py::test_add_qnode_bad_parameters PASSED             [ 48%]
test_ARAX_messenger.py::test_add_qedge_multitest PASSED                  [ 48%]
test_ARAX_overlay.py::test_jaccard PASSED                                [ 48%]
test_ARAX_overlay.py::test_add_node_pmids PASSED                         [ 49%]
test_ARAX_overlay.py::test_compute_ngd_virtual PASSED                    [ 49%]
test_ARAX_overlay.py::test_compute_ngd_attribute PASSED                  [ 50%]
test_ARAX_overlay.py::test_FET_ex1 PASSED                                [ 50%]
test_ARAX_overlay.py::test_FET_ex2 SKIPPED (need --runslow option to...) [ 50%]
test_ARAX_overlay.py::test_paired_concept_frequency_virtual SKIPPED      [ 51%]
test_ARAX_overlay.py::test_paired_concept_frequency_attribute SKIPPED    [ 51%]
test_ARAX_overlay.py::test_observed_expected_ratio_virtual SKIPPED (...) [ 51%]
test_ARAX_overlay.py::test_observed_expected_ratio_attribute SKIPPED     [ 52%]
test_ARAX_overlay.py::test_chi_square_virtual SKIPPED (need --runslo...) [ 52%]
test_ARAX_overlay.py::test_chi_square_attribute SKIPPED (need --runs...) [ 53%]
test_ARAX_overlay.py::test_predict_drug_treats_disease_virtual SKIPPED   [ 53%]
test_ARAX_overlay.py::test_predict_drug_treats_disease_attribute SKIPPED [ 53%]
test_ARAX_overlay.py::test_issue_832 SKIPPED (retire DTD)                [ 54%]
test_ARAX_overlay.py::test_issue_832_non_drug SKIPPED (retire DTD)       [ 54%]
test_ARAX_overlay.py::test_issue_840 SKIPPED (need --runslow option ...) [ 54%]
test_ARAX_overlay.py::test_issue_840_non_drug SKIPPED (need --runslo...) [ 55%]
test_ARAX_overlay.py::test_issue_892 SKIPPED (retire DTD)                [ 55%]
test_ARAX_overlay.py::test_overlay_exposures_data_virtual SKIPPED (n...) [ 56%]
test_ARAX_overlay.py::test_overlay_exposures_data_attribute SKIPPED      [ 56%]
test_ARAX_overlay.py::test_overlay_clinical_info_no_ids SKIPPED (nee...) [ 56%]
test_ARAX_overlay.py::test_missing_ngd_pmids SKIPPED (need --runslow...) [ 57%]
test_ARAX_overlay.py::test_jaccard_not_above_1 SKIPPED (need --runsl...) [ 57%]
test_ARAX_overlay.py::test_ngd_sqlite_syntax_error SKIPPED (need --r...) [ 57%]
test_ARAX_query.py::test_query_by_query_graph_2 PASSED                   [ 58%]
test_ARAX_resultify.py::test01 PASSED                                    [ 58%]
test_ARAX_resultify.py::test02 PASSED                                    [ 59%]
test_ARAX_resultify.py::test03 PASSED                                    [ 59%]
test_ARAX_resultify.py::test04 PASSED                                    [ 59%]
test_ARAX_resultify.py::test05 PASSED                                    [ 60%]
test_ARAX_resultify.py::test07 PASSED                                    [ 60%]
test_ARAX_resultify.py::test08 PASSED                                    [ 60%]
test_ARAX_resultify.py::test09 SKIPPED (need --runslow option to run)    [ 61%]
test_ARAX_resultify.py::test10 PASSED                                    [ 61%]
test_ARAX_resultify.py::test_example1 SKIPPED (need --runslow option...) [ 62%]
test_ARAX_resultify.py::test_bfs PASSED                                  [ 62%]
test_ARAX_resultify.py::test_bfs_in_essence_code PASSED                  [ 62%]
test_ARAX_resultify.py::test_issue680 SKIPPED (need --runslow option...) [ 63%]
test_ARAX_resultify.py::test_issue686a PASSED                            [ 63%]
test_ARAX_resultify.py::test_issue686b PASSED                            [ 64%]
test_ARAX_resultify.py::test_issue686c PASSED                            [ 64%]
test_ARAX_resultify.py::test_issue687 PASSED                             [ 64%]
test_ARAX_resultify.py::test_issue727 PASSED                             [ 65%]
test_ARAX_resultify.py::test_issue731 PASSED                             [ 65%]
test_ARAX_resultify.py::test_issue731b SKIPPED (need --runslow optio...) [ 65%]
test_ARAX_resultify.py::test_issue731c PASSED                            [ 66%]
test_ARAX_resultify.py::test_issue740 PASSED                             [ 66%]
test_ARAX_resultify.py::test_issue692 PASSED                             [ 67%]
test_ARAX_resultify.py::test_issue692b PASSED                            [ 67%]
test_ARAX_resultify.py::test_issue720_1 SKIPPED (need --runslow opti...) [ 67%]
test_ARAX_resultify.py::test_issue720_2 SKIPPED (need --runslow opti...) [ 68%]
test_ARAX_resultify.py::test_issue720_3 SKIPPED (need --runslow opti...) [ 68%]
test_ARAX_resultify.py::test_issue833_extraneous_intermediate_nodes PASSED [ 68%]
test_ARAX_resultify.py::test_single_node PASSED                          [ 69%]
test_ARAX_resultify.py::test_parallel_edges_between_nodes PASSED         [ 69%]
test_ARAX_resultify.py::test_issue912_clean_up_kg PASSED                 [ 70%]
test_ARAX_resultify.py::test_issue1119_a SKIPPED (need --runslow opt...) [ 70%]
test_ARAX_resultify.py::test_issue1119_b SKIPPED (need --runslow opt...) [ 70%]
test_ARAX_resultify.py::test_issue1119_c SKIPPED (need --runslow opt...) [ 71%]
test_ARAX_resultify.py::test_issue1119_d SKIPPED (need --runslow opt...) [ 71%]
test_ARAX_resultify.py::test_issue1146_a SKIPPED (need --runslow opt...) [ 71%]
test_ARAX_resultify.py::test_disconnected_qg PASSED                      [ 72%]
test_ARAX_resultify.py::test_recompute_qg_keys PASSED                    [ 72%]
test_ARAX_resultify.py::test_multi_node_edgeless_qg PASSED               [ 73%]
test_ARAX_resultify.py::test_issue_1446 SKIPPED (need --runslow opti...) [ 73%]
test_ARAX_resultify.py::test_issue_1848 SKIPPED (need --runslow opti...) [ 73%]
test_ARAX_resultify.py::test_node_binding_query_id_one_hop_single_input_curie PASSED [ 74%]
test_ARAX_resultify.py::test_node_binding_query_id_one_hop_multiple_input_curies PASSED [ 74%]
test_ARAX_resultify.py::test_node_binding_query_id_two_hop_double_pinned SKIPPED [ 75%]
test_ARAX_resultify.py::test_missing_chp_results SKIPPED (need --run...) [ 75%]
test_ARAX_resultify.py::test_too_few_results SKIPPED (need --runslow...) [ 75%]
test_ARAX_resultify.py::test_issue1923_multiple_essence_candidates_subclass SKIPPED [ 76%]
test_ARAX_standup_queries.py::test_gene_to_pathway_issue_9 PASSED        [ 76%]
test_ARAX_standup_queries.py::test_chemicals_to_gene_issue_10 PASSED     [ 76%]
test_ARAX_standup_queries.py::test_named_thing_associated_with_acrocynaosis_issue_12 PASSED [ 77%]
test_ARAX_standup_queries.py::test_chemical_substances_correlated_with_asthma_issue_18 SKIPPED [ 77%]
test_ARAX_standup_queries.py::test_diseases_treated_by_drug_issue_20 SKIPPED [ 78%]
test_ARAX_standup_queries.py::test_chemical_substances_that_down_regulate_STK11_issue_28 SKIPPED [ 78%]
test_ARAX_standup_queries.py::test_phenotypes_for_angel_shaped_phalango_epiphyseal_dysplasia_issue_33 SKIPPED [ 78%]
test_ARAX_synonymizer.py::test_example_6b PASSED                         [ 79%]
test_ARAX_synonymizer.py::test_example_9 PASSED                          [ 79%]
test_ARAX_synonymizer.py::test_example_10 PASSED                         [ 79%]
test_ARAX_synonymizer.py::test_example_11 PASSED                         [ 80%]
test_ARAX_synonymizer.py::test_example_12 PASSED                         [ 80%]
test_ARAX_synonymizer.py::test_get_canonical_curies_simple PASSED        [ 81%]
test_ARAX_synonymizer.py::test_get_canonical_curies_single_curie PASSED  [ 81%]
test_ARAX_synonymizer.py::test_get_canonical_curies_unrecognized PASSED  [ 81%]
test_ARAX_synonymizer.py::test_get_canonical_curies_by_names PASSED      [ 82%]
test_ARAX_synonymizer.py::test_get_canonical_curies_single_name PASSED   [ 82%]
test_ARAX_synonymizer.py::test_get_canonical_curies_by_names_and_curies PASSED [ 82%]
test_ARAX_synonymizer.py::test_get_canonical_curies_return_all_categories PASSED [ 83%]
test_ARAX_synonymizer.py::test_get_equivalent_nodes PASSED               [ 83%]
test_ARAX_synonymizer.py::test_get_equivalent_nodes_by_name PASSED       [ 84%]
test_ARAX_synonymizer.py::test_get_equivalent_nodes_by_curies_and_names PASSED [ 84%]
test_ARAX_synonymizer.py::test_get_normalizer_results PASSED             [ 84%]
test_ARAX_synonymizer.py::test_improper_curie_prefix_capitalization PASSED [ 85%]
test_ARAX_synonymizer.py::test_approximate_name_based_matching PASSED    [ 85%]
test_ARAX_synonymizer.py::test_entity_controller_input_no_format PASSED  [ 85%]
test_ARAX_synonymizer.py::test_entity_controller_input_minimal_format PASSED [ 86%]
test_ARAX_synonymizer.py::test_cluster_graphs PASSED                     [ 86%]
test_ARAX_translate.py::test_lookup PASSED                               [ 87%]
test_ARAX_translate.py::test_fill_success PASSED                         [ 87%]
test_ARAX_translate.py::test_fill_error PASSED                           [ 87%]
test_ARAX_translate.py::test_score PASSED                                [ 88%]
test_ARAX_translate.py::test_bind PASSED                                 [ 88%]
test_ARAX_translate.py::test_complete_results PASSED                     [ 89%]
test_ARAX_translate.py::test_filter_results_top_n PASSED                 [ 89%]
test_ARAX_translate.py::test_overlay_after_lookup PASSED                 [ 89%]
test_ARAX_translate.py::test_connect_knodes_2_nodes SKIPPED (need --...) [ 90%]
test_ARAX_translate.py::test_connect_knodes_3_nodes SKIPPED (need --...) [ 90%]
test_ARAX_translate.py::test_unknown_operation PASSED                    [ 90%]
test_ARAX_workflows.py::test_option_group_id PASSED                      [ 91%]
test_ARAX_workflows.py::test_exclude PASSED                              [ 91%]
test_ARAX_workflows.py::test_example_2 SKIPPED (need --runslow optio...) [ 92%]
test_ARAX_workflows.py::test_example_3 SKIPPED (need --runslow optio...) [ 92%]
test_ARAX_workflows.py::test_FET_example_1 SKIPPED (need --runslow o...) [ 92%]
test_ARAX_workflows.py::test_FET_example_2 SKIPPED (FW: This now run...) [ 93%]
test_ARAX_workflows.py::test_FET_example_3 SKIPPED (need issue#846 t...) [ 93%]
test_ARAX_workflows.py::test_FET_example_4 SKIPPED (need --runslow o...) [ 93%]
test_ARAX_workflows.py::test_FET_ranking_1 SKIPPED (need --runslow o...) [ 94%]
test_ARAX_workflows.py::test_example_2_kg2 SKIPPED (need --runslow o...) [ 94%]
test_ARAX_workflows.py::test_clinical_overlay_example1 SKIPPED (need...) [ 95%]
test_ARAX_workflows.py::test_clinical_overlay_example2 SKIPPED (redu...) [ 95%]
test_ARAX_workflows.py::test_two_hop_based_on_types_1 SKIPPED (redun...) [ 95%]
test_ARAX_workflows.py::test_one_hop_based_on_types_1 SKIPPED (need ...) [ 96%]
test_ARAX_workflows.py::test_one_hop_kitchen_sink_BTE_1 SKIPPED (Wor...) [ 96%]
test_ARAX_workflows.py::test_one_hop_kitchen_sink_BTE_2 SKIPPED (Wor...) [ 96%]
test_ARAX_workflows.py::test_FET_ranking_2 PASSED                        [ 97%]
test_ARAX_workflows.py::test_genetics_kp_ranking SKIPPED (need --run...) [ 97%]
test_ARAX_workflows.py::test_ranker_float_error_ex1 SKIPPED (need --...) [ 98%]
test_ARAX_workflows.py::test_ranker_float_error_ex2 SKIPPED (need --...) [ 98%]
test_ARAX_workflows.py::test_cmap_ranking SKIPPED (need --runexterna...) [ 98%]
test_ARAX_workflows.py::test_ranker_float_error_ex3 SKIPPED (need --...) [ 99%]
test_ARAX_workflows.py::test_issue_1848 SKIPPED (need --runexternal ...) [ 99%]
test_ARAX_workflows.py::test_example_3_issue_679 SKIPPED (need --run...) [100%]

=============================== warnings summary ===============================
saramsey commented 1 year ago

In order to perform this test:

Screenshot 2023-08-17 at 1 37 06 PM

here are the local code commits in the devED area:

Screenshot 2023-08-17 at 1 39 33 PM

All tests still passing:

============================================= 135 passed, 129 skipped, 52979 warnings in 306.35s (0:05:06) ==============================================
saramsey commented 1 year ago

So there were new Python requirements for ARAX, which needed to be installed before I can run the pytest suite on cicd.rtx.ai. Now the tests are running smoothly.

saramsey commented 1 year ago

I have pushed the latest master code (with the change to config_dbs.json per commit ba93dbd) to all five devareas on arax.ncats.io and restarted services: devED, kg2beta, test, beta, and devLM. Also pushed to cicd.rtx.ai.

saramsey commented 1 year ago

all tests passing on cicd.rtx.ai:

=========================== 135 passed, 129 skipped, 52595 warnings in 300.84s (0:05:00) ============================
saramsey commented 1 year ago

Confirmed, the updated kg2.ci.transltr.io is up and running and querying kg2cploverdb.ci.transltr.io:

Screenshot 2023-08-17 at 2 14 42 PM
saramsey commented 1 year ago

And the updated arax.ci.transltr.io is up and running and querying kg2.ci.transltr.io:

Screenshot 2023-08-17 at 2 16 08 PM
saramsey commented 1 year ago

For the last checklist item, I'm going to turn off our self-hosted PloverDB/KG2.8.4c endpoint (kg2cplover2.rtx.ai) and make sure nothing breaks:

ubuntu@ip-172-31-52-1:~$ sudo docker container ls -a
CONTAINER ID   IMAGE      COMMAND                  CREATED        STATUS        PORTS                                            NAMES
cf442ffb264b   ploverdb   "/entrypoint.sh /sta…"   30 hours ago   Up 30 hours   443/tcp, 0.0.0.0:9990->80/tcp, :::9990->80/tcp   ploverdb
ubuntu@ip-172-31-52-1:~$ sudo docker stop ploverdb
ploverdb
saramsey commented 1 year ago

Confirmed still working even through the PloverDB services on kg2cplover2.rtx.ai and on kg2cplover.rtx.ai are both down:

  1. arax.ncats.io/kg2beta (via the browser UI)
  2. kg2.ci.transltr.io (via the browser UI)
  3. cicd.rtx.ai (pytest suite, test_ARAX_expand.py)
  4. arax.ncats.io (via the browser UI; wouldn't expect any effect here though)
  5. arax.ncats.io/kg2 (via the browser UI; wouldn't expect any effect here though)
  6. arax.ncats.io/devED (via the browser UI; this is expected since the RTX-KG2 KP endpoints are all working, but still good to verify)
  7. arax.ncats.io/devLM (via the browser UI; this is expected since the RTX-KG2 KP endpoints are all working, but still good to verify)
  8. arax.ncats.io/beta (via the browser UI; this is expected since the RTX-KG2 KP endpoints are all working, but still good to verify)
  9. arax.ncats.io/test (via the browser UI; this is expected since the RTX-KG2 KP endpoints are all working, but still good to verify)
  10. arax.test.transltr.io (via the browser UI; this is expected since the RTX-KG2 KP endpoints are all working, but still good to verify)
  11. arax.transltr.io (via the browser UI; this is expected since the RTX-KG2 KP endpoints are all working, but still good to verify)
  12. kg2.test.transltr.io (via the browser UI)
  13. kg2.transltr.io (via the browser UI)
  14. arax.ci.transltr.io (via the browser UI; this is expected since the RTX-KG2 KP endpoints are all working, but still good to verify)
saramsey commented 1 year ago

Now we are ready to do a direct comparison of the results of running Sundar's test TRAPI query,

{
   "edges": {
      "e00": {
         "subject":   "n00",
         "object":    "n01",
         "predicates": ["biolink:diagnoses"]
      }
   },
   "nodes": {
      "n00": {
         "ids":        ["UMLS:C1510438"]
      },
      "n01": {
         "categories":  ["biolink:Disease"]
      }
   }
}

on kg2.ci.transltr.io and on kg2.test.transltr.io.

On the TEST instance, we get:

Screenshot 2023-08-17 at 2 41 12 PM

and on the CI instance, we get:

Screenshot 2023-08-17 at 2 41 31 PM

and on the CI instance, we also see this message explaining why there are no results:

Screenshot 2023-08-17 at 2 41 50 PM
saramsey commented 1 year ago

Since the "hold your breath" tests passed, I've stopped the kg2cplover2.rtx.ai instance, since nothing is using it.

saramsey commented 1 year ago

Since the last three remaining tasks are not critical and have been broken out into separate issues, I'm going to close out this issue.

saramsey commented 1 year ago

still need to merge to itrb-test; I think that step maybe needs to be added to the checklist

saramsey commented 1 year ago

merge done for PloverDB (see RTXteam/PloverDB PR 49)

saramsey commented 1 year ago

merge done for RTX (see RTXteam/RTX PR 2105)

saramsey commented 1 year ago

We plan to ask Sarah Stemann, to open a ticket for deploying to ITRB TEST tomorrow morning (Friday Aug. 18). @edeutsch has a code update for TRAPI validation, that he wants to include. Aiming to send that request to Sarah tonight.

saramsey commented 1 year ago

RTX master has been merged to production (see RTXteam/RTX PR 2107).

saramsey commented 1 year ago

Deployment to ITRB TEST confirmed working.

saramsey commented 1 year ago

We have messaged Sarah Stemann and ITRB to request deployment to ITRB PROD.

saramsey commented 1 year ago

For both RTXteam/RTX and RTXteam/PloverDB, I have tagged the code that was merged to production with the git tag augfreeze-rc (here, "rc" stands for "release candidate").

For RTX: https://github.com/RTXteam/RTX/releases/tag/augfreeze-rc

For PloverDB: https://github.com/RTXteam/PloverDB/releases/tag/augfreeze-rc

saramsey commented 11 months ago

Thank you @sundareswarpullela !