Tests fail on test instance

[edeutsch]

Although tests seem to pass locally, it seems that commits cause a test failure on the automatic testing instance:

https://github.com/RTXteam/RTX/actions/runs/7225709946/job/19689713862

[saramsey]

Error seems to be:

FAILED test_ARAX_expand.py::test_edge_constraints - AssertionError: assert ('...
==== 2 failed, 133 passed, 128 skipped, 36249 warnings in 753.67s (0:12:33) ====
Error: Process completed with exit code 1.

[amykglen]

so @sundareswarpullela figured out a fix for this issue!

though we were having trouble seeing why that fix works, so we spent some time today digging into that to try to understand what's going on (seems a little suspicious).

latest update (mostly for @sundareswarpullela):

so the failing test (test_edge_constraints) does successfully grab KG2 from the KP info cache:

2023-12-22T01:04:45.857612 DEBUG: (206) [] valid kps: {'infores:spoke', 'infores:automat-ubergraph', 'infores:automat-viral-proteome', 'infores:molepro', 'infores:automat-hgnc', 'infores:cohd', 'infores:automat-gtex', 'infores:automat-robokop', 'infores:connections-hypothesis', 'infores:agrkb', 'infores:knowledge-collaboratory', 'infores:text-mining-provider-cooccurrence', 'infores:automat-intact', 'infores:automat-panther', 'infores:gelinea', 'infores:automat-icees-kg', 'infores:genetics-data-provider', 'infores:automat-hmdb', 'infores:automat-cam-kp', 'infores:automat-ctd', 'infores:automat-drug-central', 'infores:service-provider-trapi', 'infores:automat-string-db', 'infores:automat-gwas-catalog', 'infores:rtx-kg2', 'infores:automat-pharos', 'infores:automat-human-goa', 'infores:automat-hetio', 'infores:openpredict', 'infores:automat-gtopdb'}

but it looks like Expand doesn't think that KG2 can answer the given query based on its meta knowledge graph:

2023-12-22T01:04:45.857647 DEBUG: (206) [] KPs that can answer this query: {'infores:automat-ctd', 'infores:spoke', 'infores:automat-ubergraph', 'infores:automat-drug-central', 'infores:service-provider-trapi', 'infores:molepro', 'infores:cohd', 'infores:automat-icees-kg', 'infores:automat-pharos', 'infores:automat-robokop', 'infores:automat-hmdb', 'infores:automat-cam-kp', 'infores:automat-hetio', 'infores:text-mining-provider-cooccurrence', 'infores:automat-gtopdb'} 2023-12-22T01:04:45.857674 DEBUG: (206) [] excluded by version: set() 2023-12-22T01:04:45.857696 DEBUG: (206) [] excluded by maturity: {'infores:automat-sri-reference-kg'}

(those log messages are from the latest test build here)

this makes me wonder whether the cicd.rtx.ai local KG2 has a functioning /meta_knowledge_graph endpoint at the time that kp_info_cacher.py is run for ARAX? if it doesn't, getting that endpoint up and working before ARAX creates the KP info cache might fix the issue?

but I still don't get why this issue isn't occurring in all branches then... 😂

[amykglen]

ok, I confirmed that KG2 doesn't have a meta map when Expand loads the KP info from the cache during the failing test:

  - 2023-12-22T02:09:31.259367 DEBUG: [] Loading cached Smart API amd meta map info
  - 2023-12-22T02:09:31.263084 DEBUG: [] KG2 meta map is: None

[amykglen]

is this still an issue? it seems like the CICD test builds have been working fine as of late with commits to master

[sundareswarpullela]

Yeah, it still seems to be an issue as of 3 weeks ago

[amykglen]

huh, so it's an intermittent issue in itrb-test?

[sundareswarpullela]

I'm re-running the CI/CD tests on itrb-test branch to see if its working now. If not, we can take a look at the logs.

[sundareswarpullela]

Weirdly it seems to have failed in the Build Docker image step:

https://github.com/RTXteam/RTX/actions/runs/9618139341/job/27341557705