Generalize Node.qnode_id and Edge.qedge_id to lists

[edeutsch]

Tried to run the following ambitious query: add_qnode(curie=DOID:14330, id=n00) add_qnode(is_set=true, id=n01) add_qnode(type=disease, id=n02) add_qedge(source_id=n00, target_id=n01, id=e00) add_qedge(source_id=n01, target_id=n02, id=e01) expand(edge_id=[e00,e01]) overlay(action=compute_jaccard, start_node_id=n00, intermediate_node_id=n01, end_node_id=n02, virtual_edge_type=J1) resultify(ignore_edge_direction=true) filter_results(action=sort_by_edge_attribute, edge_attribute=jaccard_index, direction=descending, max_results=50)

The query poops out at resultify() with: DEBUG: Processing action 'resultify' with parameters {'ignore_edge_direction': 'true'}
ERROR: The two nodes for KG edge 1281618, DOID:0050890 and DOID:1289, have no corresponding edge in the QG
DEBUG: Applying Resultifier to Message with parameters {'ignore_edge_direction': 'true'}

Unclear if this is aproblem with resultify() or if resultify is legitimately unhappy because expand() has a bug.

[amykglen]

Think I know what's causing this error:

During expansion of your query graph, node DOID:14330 (corresponding to n00) is also being returned from neo4j as a result for n02 (generic disease). Therefore, this node technically has multiple qnode_ids, but we only store one of those on the node. (This was a corner case we discussed in the past but decided not to worry about at the time, as it seemed unlikely to be encountered.)

I adjusted expand to throw an error when this happens (for now at least). Not sure if we want to come up with a way to allow multiple qnode/qedge IDs?

[edeutsch]

Ah, excellent, good sleuthing! Definitely a good topic for discussion. For this type of query, including the n00 node as a n02 is sort of a degenerate solution (i.e. the disease most similar to DOID:14330 is of course DOID:14330, but not interesting) Seems like we want to force DOID:14330 to remain a n00 and be excluded from being a n02.

[amykglen]

True - for that query, it doesn't seem helpful to have DOID:14330 appear as an n02...

Though for a query like this, for example:

(n00 {id:"DOID:14330"})--(n01:protein)--(n02:chemical_substance)--(n03:protein)

it seems these would both be relevant results:

(n00 DOID:14330)--(n01 synuclein alpha)--(n02 cilnidipine)--(n03 histone deacetylase 8)
(n00 DOID:14330)--(n01 monoamine oxidase A)--(n02 cilnidipine)--(n03 synuclein alpha)

which means synuclein alpha would technically have two qnode_ids (n01 and n03).

Not sure how often/if such a query would be run in practice though.

[edeutsch]

Which proteins interact with drugs that interact with proteins that are related to fanconi anemia? Seems like a reasonable question.

I think our data model will break down with this. Having the same node with the same CURIE play two different roles isn't going to work so well. This is something that we should consider with the whole team and probably even Translator-wide. One can imagine a hack of adding an underscore to the end of the CURIE to make it different in the different roles.

So, probably not often, but it seems like something that we should plan for.

[amykglen]

Sounds good to discuss with the team, though I'm not sure this is a Translator-wide issue? I think the ReasonerStdAPI already allows for this, since the mapping of qnode_id -> node doesn't happen within the knowledge graph, but in the result bindings. Meaning, different result bindings could refer to synuclein alpha for different query nodes, but synuclein alpha still only exists once in the knowledge graph.

So I think the challenge is more how we want to deal with this internally, since we've opted to decorate nodes with their qnode_ids for our own purposes.

[edeutsch]

yeah, I think you're right. So maybe internally we just need to allow qnode_id be a map of multiple ids so that Expand() can communicate to Resultify() when the came CURIE plays two roles.

[saramsey]

It sounds like we are contemplating a change to the Reasoners Standard API object model such that a Node object can have a one-to-many relationship to QNodes (correct?). Here is where we are at right now: currently, the _get_results_for_kg_by_qg function (which is the actual internal function that does the "resultifying") in ARAX_resultify.py(line 369) requires that any Node object have one (and only one) qnode_id. If not, the function raises a ValueError. If that mapping becomes one-to-many, the internal map node_bindings_map would have to become one-to-many. This will complicate the procedure for ascertaining whether or not a given set of KG nodes can define a subgraph that matches the QG (previously, with a one-to-one mapping, it was trivial). But I think that it should be doable, provided that the cardinality of the number of nodes in the QG never gets too big. Tagging @dkoslicki for his graph insights here.

[saramsey]

Why can't this query be handled with a query graph like this?

QNodes:

• n0: protein
• n1: drug
• n2: protein (is_set=True)

QEdges:

• n0 -[interacts_with]- n1
• n1 -[interacts_with]- n2
• n2 -[associated_with]- MONDO:0019391

[edeutsch]

MONDO:0019391 has to be a node, so like n03

[edeutsch]

and there's no change to the standard, but rather just our internal convenience feature qnode_id in KG Node()

[amykglen]

yeah, I think you're right. So maybe internally we just need to allow qnode_id be a map of multiple ids so that Expand() can communicate to Resultify() when the came CURIE plays two roles.

Yeah - and I think overlay relies on QG IDs as well (not just expand and resultify).

One crazy alternative idea to annotating nodes/edges with their list of QG IDs is to instead (internally) organize them by their QG IDs, like:

• knowledge_graph:
  • nodes:
  • n00: [Node1]
  • n01: [Node2]
  • n02: [Node3, Node1]
  • edges:
  • e00: [Edge1]
  • e01: [Edge2, Edge3]

So you can see Node1 appears under both n00 and n02.

This is really nice/more performant for expand, as I almost always am trying to grab nodes/edges with a particular qnode_id/qedge_id, but not sure how well it would work for other modules. (Maybe help? Maybe hurt?) Doubt it's something we'd choose to pursue, but thought I'd throw it out there.

[edeutsch]

That's crazy!

And I love it!

But not for me to decide since I don't have much to do with code that would be affected by this. The writers of Expand, Overlap, and Resultify would need to jointly decide whether this is a useful/better internal representation.

[saramsey]

and there's no change to the standard, but rather just our internal convenience feature qnode_id in KG Node()

I'm afraid I don't understand this. The node object clearly has a qnode_id property as shown in this YAML file:

https://github.com/RTXteam/RTX/blob/master/code/UI/OpenAPI/python-flask-server/RTX_0.9.3_Swagger2.0.yaml

and isn't this YAML file the embodiment of the reasoner's API standard?

Or are you saying that this YAML file has extra stuff that is not specified in the standard?

[edeutsch]

This is our implementation of the standard. Note that the official "standard" is still at 0.9.2, is on OpenAPI 3.0.1, not Swagger 2.0, and has both a core schema as well as an "experimental/optional" branch that individual sites may implement. Plus we added some things for our own convenience. Like qnode_id and log[ ]. So, that YAML file that you reference is our embodiment of what we chose to implement that is mostly the core plus the experimental plus some new spicy things that we've thrown into the soup. The real standard is here: https://github.com/NCATS-Tangerine/NCATS-ReasonerStdAPI/blob/master/API/TranslatorReasonersAPI.yaml It's missing a lot of fun stuff. Everyone is more or less doing the same things, which is why the discussion around this topic is so complex.

[saramsey]

@edeutsch thank you for the very helpful explanation

[saramsey]

Think I know what's causing this error:

During expansion of your query graph, node DOID:14330 (corresponding to n00) is also being returned from neo4j as a result for n02 (generic disease). Therefore, this node technically has multiple qnode_ids, but we only store one of those on the node. (This was a corner case we discussed in the past but decided not to worry about at the time, as it seemed unlikely to be encountered.)

Amy and Eric, I had always assumed that a node in the KG cannot match both a fixed node in the QG (e.g., "DOID:14330") and a non-fixed node in the QG (or more generally, two distinct nodes in the QG at the same time). Consider a simple example with query graph:

qnode(curie=DOID:14330, id=n00)
qnode(type=protein, id=n01)
qnode(type=disease, id=n02)
qedge(source_id=n00, target_id=n01)
qedge(source_id=n01, target_id=n02)

please correct me if I am wrong, but it sounds like in response to this issue we are proposing to allow matching the above QG with a KG subgraph like this

node(curie=DOID:14330)
node(curie=UniProtKB:12345)
edge(source_id=DOID:14330, target_id=UniProtKB:12345)
edge(source_id=UniProtKB:12345, target_id=DOID:14330)

Is that correct? If so, I don't see why the proposed KG should be allowed to match the QG. It's not a subgraph isomorphism (right, @dkoslicki ?). The above KG subgraph has two nodes and the QG has three nodes. If I as the user request a query graph with three nodes, I feel like it is reasonable to expect each result to have three QG-mapped nodes in it. Also, allowing this kind of match would require some changes in ARAX_resultify, and I'm not so clear on why we want to allow it (see below).

As for this query (I will call it "Query 2"):

(n00 {id:"DOID:14330"})--(n01:protein)--(n02:chemical_substance)--(n03:protein)

I'm not sure why we would want results where n01==n03. But suppose that for some reason we did. And suppose we had modified Expander so that Query 2 does not error out and does not return any results where n01==n03. It seems to me that we could easily get results where n01==n03 (along with all the results where n01<>n03) by constructing the union set of proteins in the following two queries:

(n00 {id:"DOID:14330"})--(n01:protein)--(n02:chemical_substance)--(n03:protein)
(n00 {id:"DOID:14330"})--(n01:protein)--(n02:chemical_substance)

If there is a demand for such a workflow, I think we should consider the possibility of implementing a "results union" in the DSL to handle it as described above. But if that is not possible and if we really need n01==n03 type results as described above, I think we should make this a conditional mode in resultify (i.e., we should have a resultify DSL parameter option strict_subgraph_isomorphism=bool, for which the default value could be a matter of discussion). My reasoning is is in part that anyone using the system who has used SPARQL before might find the n01==n03 result quite surprising; thus having a strict_subgraph_isomorphism=true mode of operation would probably be best for people who want to think of their query graphs with a "SPARQL"-like mental model.

Sorry to be a sticky wicket about this, but ARAX_resultify was carefully constructed under the interface contract that each KG node maps to one and only one QG node. To change that means (I think) a fairly significant alteration of the resultify module. So while that is not necessarily an argument for not ever changing it, it is an argument for carefully considering what has changed in our use-cases to warrant breaking the interface's data model.

[edeutsch]

I don't have an answer, there are several possible approaches here. But I think this is a very important topic for discussion. In an ideal world I would suggest that @saramsey , @dkoslicki , @amykglen , and I (and anyone else) all sit together in a room with a whiteboard for 2 hours and hash this out and make decisions.

[saramsey]

OK next week works for me. This week is impossible.

[amykglen]

Just to clarify about this part of your response, @saramsey:

I'm not sure why we would want results where n01==n03.

I think with "Query 2" I was pointing out a slightly different issue than this - in my example, both results have four unique nodes (so within each result n01<>n03), but one result uses a particular protein (synuclein alpha) as n01, and the other result uses that same protein as n03.

[saramsey]

Just to clarify about this part of your response, @saramsey:

I'm not sure why we would want results where n01==n03.

I think with "Query 2" I was pointing out a slightly different issue than this - in my example, both results have four unique nodes (so within each result n01<>n03), but one result uses a particular protein (synuclein alpha) as n01, and the other result uses that same protein as n03.

Ah, I missed that. Thank you @amykglen. OK, I am starting to come around to the idea of making the qnode_id field of the Node object a list. No doubt a bunch of code in resultify would need to be fixed to accommodate this.

[saramsey]

If we do make qnode_id a list, now we'll have to loop over all possible KG to QG mappings. And they can't vary independently; i.e., if we have proteins foo and bar in the KG and both can map to n02 or n04 in the QG, then we have to enumerate the two cases

foo: n02
bar: n04

and

foo: n04
bar: n02

while disallowing cases such as

foo: n04
bar: n04

[amykglen]

Yeah... you mean within the context of a single result?

Personally I still like the idea of letting go of the qnode_id/qedge_id properties and instead organizing the knowledge graph by qg ids (described here). So the data structure would look like:

• knowledge_graph:
  • nodes:
  • n00: [Node1, ...]
  • n01: [Node2, ...]
  • n02: [Node3, Node1, ...]
  • edges:
  • e00: [Edge1, ...]
  • e01: [Edge2, Edge3, ...]

I'm actually using that structure within expand now... but I'm fine with whatever method!

[saramsey]

Yeah... you mean within the context of a single result?

Personally I still like the idea of letting go of the qnode_id/qedge_id properties and instead organizing the knowledge graph by qg ids (described here). So the data structure would look like:

• knowledge_graph:

  • nodes:

  • n00: [Node1, ...]

  • n01: [Node2, ...]

  • n02: [Node3, Node1, ...]

  • edges:

  • e00: [Edge1, ...]

  • e01: [Edge2, Edge3, ...]

I'm actually using that structure within expand now... but I'm fine with whatever method!

I'm open to trying this. A few thoughts:

1. I think we should do this in a branch. This work might take a while and I expected a period of semi-breakage. Would be nice to share code back and forth in this branch without affecting master.
2. I think mypy validation will be very helpful to making the transition by reducing the amount of run-time debugging that must be done. Thus I implemented #752 as a first step.
3. A bunch of test case code will be broken by this, and will also need to get fixed. See ARAX_resultify_testcases.py which (as of completion of #751) contains all of the unittest cases for the module ARAX_resultify.py. Fixing all 26 test cases will take some time.

[amykglen]

Sounds good to me! Changes should be minimal for expand since it's already using this structure internally (and for testing), so I'm happy to help with other modules as desired.

This would also affect modules other than just expand and resultify, of course, (overlay, filter) so I suppose it would be good to confirm with @dkoslicki and @finnagin that this plan is ok?

[dkoslicki]

Currently, overlay and filter make use of qedge_id and qnode_id (eg here, here, here, here these show up all over the place), so working in a branch is good, as well as thinking through the value add of this architectural change and how it affects other ARAX components. If this is only relevant to resultify and/or expand, then my concerns are a non-issue.

[amykglen]

We decided to go with the method of making Node.qnode_id and Edge.qedge_id lists during minihackathon on 5/29

All work will be done in branch(es): I'll make the change to Expand and then let @dkoslicki know so he can update Overlay; I'll also make the changes to Resultify.

[amykglen]

@edeutsch - is turning qnode_id/qedge_id into lists in our Swagger model just a matter of updating this file? (starting to work on this issue in branch issue720)

[edeutsch]

formally it is not that simple. That file is used for code generation to generate the classes that we use. However, I think I've learned that the classes are pretty laissez faire about what data types you use. They do not really enforce data types. So, I suspect that informally no change is even required. You can just start doing it and the autogen classes won't care. The only sticking point is that resultify() will need to be updated to handle the list() case at the same time. I'm not 100% sure of that answer, but I think it is correct.

[amykglen]

Ok, yeah, I've started by just populating the qnode_id/qedge_id properties with lists in Expand (in a branch, so I don't have to rush to update Resultify or anything), so that is functionally what we want - but do you think the names of these fields should be made plural (qnode_ids/qedge_ids) since they'll be lists now? (Which would require updating the autogen classes)

[edeutsch]

I don't have much of an opinion. Since it is just an internal thing, I don't think it really matters. Are you thinking of usually having it be a scalar, but then switching to a list when needed? Or always making it a list, usually a list of one? If the former, I would just leave it. If the latter, probably better to rename it.

This does remind me that I was once thinking that maybe we ought to have specialized code in resultify to delete those keys once it's done with them. It really shouldn't be in the output since it's not part of the spec. Although, of course, extra attributes are permitted. But they are redundant in this case.

[amykglen]

I was planning to always have them be lists.

And yeah, does make sense to delete them for the output.

[edeutsch]

okay, if always lists, then making it plural sounds tidier. I am uncertain if the classes as currently generated would let you store a new attribute that the plural name. If they do, then you can just start using it. If not, then we would need to update the yaml and regenerate that class.

[amykglen]

Cool. Ok, tried out what you said and it does seem to let me store attributes by the plural names even though they don't exist in the yaml file/classes, but for some reason when I print the knowledge graph, these attributes do not appear, and the old qnode_id/qedge_id do (set to None):

{'description': 'AGALSIDASE ALFA; MAX_FDA_APPROVAL_PHASE: 4',
 'id': 'CHEMBL.COMPOUND:CHEMBL2108214',
 'name': 'AGALSIDASE ALFA',
 'node_attributes': [{'name': 'synonym',
                      'type': None,
                      'url': None,
                      'value': ['Agalsidase Alfa',
                                'AGALSIDASE ALFA',
                                'USP/USAN:5020',
                                'ATC:5020',
                                'BNF:5020']},
                     {'name': 'category',
                      'type': None,
                      'url': 'http://w3id.org/biolink/vocab/Chemical_Substance',
                      'value': None},
                     {'name': 'provided_by',
                      'type': None,
                      'url': 'https://www.ebi.ac.uk/chembl',
                      'value': None},
                     {'name': 'deprecated',
                      'type': None,
                      'url': None,
                      'value': False},
                     {'name': 'update_date',
                      'type': None,
                      'url': None,
                      'value': '2018-12-10'}],
 'qnode_id': None,
 'symbol': None,
 'type': ['chemical_substance'],
 'uri': 'https://www.ebi.ac.uk/chembl/compound/inspect/CHEMBL2108214'}

The plural attributes do seem to actually exist on the nodes, however, because hasattr(node, 'qnode_ids') returns True and Expand still works. They're just invisible somehow. I think it would be cleaner to update the yaml file/regenerate classes. (Though doing so is not blocking me from working on this issue.)

[edeutsch]

I suspect that the serializer is part of the auto-generated code, and so if it isn't in the yaml, then it doesn't get serialized. Maybe we can exploit that by not having any yaml entry at all for these items. We can store the values however we like, we can access them and use them, but they're never serialized back to the client, so we don't even have to worry about deleting them.

[amykglen]

That sounds fine to me. I guess the only potential downsides I can think of are in documentation/debugging - everyone working on the code would have to somehow know that these attributes exist, even though they're not in the yaml file (the typical place to look to see what attributes exist). And everyone would need to be aware that if they want to actually see a node's/edge's qnode_ids/qedge_ids, they have to explicitly print that field (rather than just the node/edge).

[amykglen]

@dkoslicki @finnagin - just letting you know I'm finished adjusting Expand here, so you can update Overlay/Filter whenever you'd like. Changes are in the branch issue720 (which you're welcome to use if you want). In summary, nodes/edges that Expand adds to the KG now have qnode_ids/qedge_ids attributes, which are always lists.

(Note that the old qnode_id and qedge_id attributes are currently still appearing on nodes/edges (but not filled out with any data), but those will go away eventually... discussion with @edeutsch about whether to rename them in or remove them from the Swagger yaml/autogen classes is above.)

(And also note that Resultify isn't yet updated, so won't work yet - only Expand. Working on Resultify next.)

[amykglen]

Another potential reason for keeping qnode_ids/qedge_ids as actual attributes in the Swagger classes is that the initializer for the classes won't accept a property not in the class. So code that looks like this:

node = Node(id=node_info['id'], type=[node_info['type']], qnode_ids=node_info['qnode_ids'])

Now throws this error:

TypeError: __init__() got an unexpected keyword argument 'qnode_ids'

So something like this would have to be done instead:

node = Node(id=node_info['id'], type=[node_info['type']])
node.qnode_ids = node_info['qnode_ids']

[amykglen]

Changes for this issue are complete for Expand and Resultify in the branch issue720 - both are passing all unit tests. Remaining modules that need updating are Overlay and Filter.

[amykglen]

Posting for the record: We decided at mini-hackathon on 6/5 to make qnode_ids and qedge_ids official attributes in the Swagger Node/Edge models (and remove the old qnode_id/qedge_id). @edeutsch is planning to do this.

[amykglen]

ok, @edeutsch has pushed the new Node/Edge classes to the issue720 branch, so final steps are for @dkoslicki /@finnagin to update Overlay/Filter to use Node.qnode_ids and Edge.qedge_ids!

[dkoslicki]

On the agenda for tomorrow (today?) to discuss how best to implement this and how it will affect overlay features

[dkoslicki]

@amykglen I think I've finished with everything on my end. All tests in test_ARAX_overlay.py and test_ARAX_workflows.py (which includes your ARAX_resultify_testcases.test_example3() test) are passing on the issue720 branch.

[amykglen]

awesome! yeah, things seem to be running great in issue720... I guess we're at the point where we can merge issue720 into master?

[dkoslicki]

Yes, I assume that's the case! @amykglen do you want to handle that merge? There may be a few conflicts wrt documentation and other minor things, so let me know if you run into any problems or would like me to handle it

[amykglen]

ok! yep, I'll give it a go.

[amykglen]

ok, so I was able to merge master into issue720 with no conflicts, but when I run pytest -v in ARAX/test/, 9 Overlay tests are failing for me... all complain about sqlite3.OperationalError: unable to open database file

I doubt this is related to the qnode_ids/qedge_ids stuff.. maybe a local problem for me? do those tests pass for anyone else in issue720 (after pulling)?

The output is huge, but here's the first several failing tests:

(rtx-env-7) amys-macbook:test aglen$ pytest -v
================================================================= test session starts =================================================================
platform darwin -- Python 3.7.6, pytest-5.4.3, py-1.8.1, pluggy-0.13.1 -- /Users/aglen/translator/RTX/rtx-env-7/bin/python3
cachedir: .pytest_cache
rootdir: /Users/aglen/translator/RTX/RTX/code/ARAX/test
collected 57 items                                                                                                                                    

test_ARAX_messenger.py::test_create_message_basic PASSED                                                                                        [  1%]
test_ARAX_messenger.py::test_create_message_node_edge_types PASSED                                                                              [  3%]
test_ARAX_messenger.py::test_add_qnode_basic PASSED                                                                                             [  5%]
test_ARAX_messenger.py::test_add_qnode_curie_scalar PASSED                                                                                      [  7%]
test_ARAX_messenger.py::test_add_qnode_curie_list PASSED                                                                                        [  8%]
test_ARAX_messenger.py::test_add_qnode_name PASSED                                                                                              [ 10%]
test_ARAX_messenger.py::test_add_qnode_type PASSED                                                                                              [ 12%]
test_ARAX_messenger.py::test_add_qnode_bad_name PASSED                                                                                          [ 14%]
test_ARAX_messenger.py::test_add_qnode_bad_parameters PASSED                                                                                    [ 15%]
test_ARAX_messenger.py::test_add_qedge_multitest PASSED                                                                                         [ 17%]
test_ARAX_overlay.py::test_jaccard PASSED                                                                                                       [ 19%]
test_ARAX_overlay.py::test_add_node_pmids FAILED                                                                                                [ 21%]
test_ARAX_overlay.py::test_compute_ngd_virtual FAILED                                                                                           [ 22%]
test_ARAX_overlay.py::test_compute_ngd_attribute FAILED                                                                                         [ 24%]
test_ARAX_overlay.py::test_FET PASSED                                                                                                           [ 26%]
test_ARAX_overlay.py::test_paired_concept_frequency_virtual FAILED                                                                              [ 28%]
test_ARAX_overlay.py::test_paired_concept_frequency_attribute FAILED                                                                            [ 29%]
test_ARAX_overlay.py::test_observed_expected_ratio_virtual FAILED                                                                               [ 31%]
test_ARAX_overlay.py::test_observed_expected_ratio_attribute FAILED                                                                             [ 33%]
test_ARAX_overlay.py::test_chi_square_virtual FAILED                                                                                            [ 35%]
test_ARAX_overlay.py::test_chi_square_attribute FAILED                                                                                          [ 36%]
test_ARAX_overlay.py::test_predict_drug_treats_disease_virtual PASSED                                                                           [ 38%]
test_ARAX_overlay.py::test_predict_drug_treats_disease_attribute PASSED                                                                         [ 40%]
test_ARAX_query.py::test_query_by_canned_query_Q0 PASSED                                                                                        [ 42%]
test_ARAX_query.py::test_query_by_query_graph_2 FAILED                                                                                          [ 43%]
test_ARAX_resultify.py::TestARAXResultify::test01 PASSED                                                                                        [ 45%]
test_ARAX_resultify.py::TestARAXResultify::test02 PASSED                                                                                        [ 47%]
test_ARAX_resultify.py::TestARAXResultify::test03 PASSED                                                                                        [ 49%]
test_ARAX_resultify.py::TestARAXResultify::test04 PASSED                                                                                        [ 50%]
test_ARAX_resultify.py::TestARAXResultify::test05 PASSED                                                                                        [ 52%]
test_ARAX_resultify.py::TestARAXResultify::test06 PASSED                                                                                        [ 54%]
test_ARAX_resultify.py::TestARAXResultify::test07 PASSED                                                                                        [ 56%]
test_ARAX_resultify.py::TestARAXResultify::test08 PASSED                                                                                        [ 57%]
test_ARAX_resultify.py::TestARAXResultify::test09 PASSED                                                                                        [ 59%]
test_ARAX_resultify.py::TestARAXResultify::test10 PASSED                                                                                        [ 61%]
test_ARAX_resultify.py::TestARAXResultify::test_bfs PASSED                                                                                      [ 63%]
test_ARAX_resultify.py::TestARAXResultify::test_bfs_in_essence_code PASSED                                                                      [ 64%]
test_ARAX_resultify.py::TestARAXResultify::test_example1 PASSED                                                                                 [ 66%]
test_ARAX_resultify.py::TestARAXResultify::test_example2 PASSED                                                                                 [ 68%]
test_ARAX_resultify.py::TestARAXResultify::test_issue680 PASSED                                                                                 [ 70%]
test_ARAX_resultify.py::TestARAXResultify::test_issue686a PASSED                                                                                [ 71%]
test_ARAX_resultify.py::TestARAXResultify::test_issue686b PASSED                                                                                [ 73%]
test_ARAX_resultify.py::TestARAXResultify::test_issue686c PASSED                                                                                [ 75%]
test_ARAX_resultify.py::TestARAXResultify::test_issue687 PASSED                                                                                 [ 77%]
test_ARAX_resultify.py::TestARAXResultify::test_issue692 PASSED                                                                                 [ 78%]
test_ARAX_resultify.py::TestARAXResultify::test_issue692b PASSED                                                                                [ 80%]
test_ARAX_resultify.py::TestARAXResultify::test_issue720_1 PASSED                                                                               [ 82%]
test_ARAX_resultify.py::TestARAXResultify::test_issue720_2 PASSED                                                                               [ 84%]
test_ARAX_resultify.py::TestARAXResultify::test_issue720_3 PASSED                                                                               [ 85%]
test_ARAX_resultify.py::TestARAXResultify::test_issue727 PASSED                                                                                 [ 87%]
test_ARAX_resultify.py::TestARAXResultify::test_issue731 PASSED                                                                                 [ 89%]
test_ARAX_resultify.py::TestARAXResultify::test_issue731b PASSED                                                                                [ 91%]
test_ARAX_resultify.py::TestARAXResultify::test_issue731c PASSED                                                                                [ 92%]
test_ARAX_resultify.py::TestARAXResultify::test_issue740 PASSED                                                                                 [ 94%]
test_ARAX_resultify.py::test_example3 FAILED                                                                                                    [ 96%]
test_ARAX_workflows.py::test_example_2 PASSED                                                                                                   [ 98%]
test_ARAX_workflows.py::test_example_3 FAILED                                                                                                   [100%]

====================================================================== FAILURES =======================================================================
_________________________________________________________________ test_add_node_pmids _________________________________________________________________

    def test_add_node_pmids():
        query = {"previous_message_processing_plan": {"processing_actions": [
            "create_message",
            "add_qnode(name=DOID:384, id=n00)",
            "add_qnode(type=chemical_substance, is_set=true, id=n01)",
            "add_qedge(source_id=n00, target_id=n01, id=e00)",
            "expand(edge_id=e00)",
            "overlay(action=add_node_pmids, max_num=15)",
            "return(message=true, store=false)"
        ]}}
        [response, message] = _do_arax_query(query)
        print(response.show())
        # check response status
>       assert response.status == 'OK'
E       AssertionError: assert 'ERROR' == 'OK'
E         - OK
E         + ERROR

test_ARAX_overlay.py:126: AssertionError
---------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------
Response:
  status: ERROR
  n_errors: 2  n_warnings: 1  n_messages: 50
  error_code: UnknownError   message: Something went wrong adding the PubMed ID attributes
  - 2020-06-11 16:07:28.499817 WARNING: Utilizing API calls to NCBI eUtils, so this may take a while...
  - 2020-06-11 16:07:28.534855 ERROR: Traceback (most recent call last):
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/test/../ARAXQuery/Overlay/add_node_pmids.py", line 45, in add_node_pmids
    pmids = NGD.get_pmids_for_all([node_curie], [node_name])[0]  # since the function was designed for multiple inputs, but I only want the first
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/NormGoogleDistance.py", line 317, in get_pmids_for_all
    terms[a] = NormGoogleDistance.get_mesh_term_for_all(curie_id_list[a], description_list[a])
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/NormGoogleDistance.py", line 169, in get_mesh_term_for_all
    if QueryNCBIeUtils.is_mesh_term(description):
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryNCBIeUtils.py", line 501, in is_mesh_term
    ret_list = QueryNCBIeUtils.get_mesh_uids_for_mesh_term(mesh_term)
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryNCBIeUtils.py", line 175, in get_mesh_uids_for_mesh_term
    'db=mesh&term=' +  urllib.parse.quote(mesh_term + '[MeSH Terms]', safe=''))
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryNCBIeUtils.py", line 64, in send_query_get
    requests = CacheControlHelper()
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/cache_control_helper.py", line 32, in __init__
    self.sess = CacheControl(requests.session(), heuristic=CustomHeuristic(days=30), cache=FileCache('.web_cache'))
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests/sessions.py", line 770, in session
    return Session()
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/core.py", line 212, in __init__
    **backend_options
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/core.py", line 73, in __init__
    self.cache = backends.create_backend(backend, cache_name, backend_options)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/__init__.py", line 63, in create_backend
    return registry[backend_name](cache_name, **options)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/sqlite.py", line 28, in __init__
    self.responses = DbPickleDict(location + extension, 'responses', fast_save=fast_save)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/storage/dbdict.py", line 62, in __init__
    with self.connection() as con:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/storage/dbdict.py", line 74, in connection
    con = sqlite.connect(self.filename)
sqlite3.OperationalError: unable to open database file

  - 2020-06-11 16:07:28.534893 ERROR: Something went wrong adding the PubMed ID attributes

______________________________________________________________ test_compute_ngd_virtual _______________________________________________________________

    def test_compute_ngd_virtual():
        query = {"previous_message_processing_plan": {"processing_actions": [
            "create_message",
            "add_qnode(name=DOID:384, id=n00)",
            "add_qnode(type=chemical_substance, is_set=true, id=n01)",
            "add_qedge(source_id=n00, target_id=n01, id=e00)",
            "expand(edge_id=e00)",
            "overlay(action=compute_ngd, source_qnode_id=n00, target_qnode_id=n01, virtual_relation_label=N1)",
            "resultify(ignore_edge_direction=true, debug=true)",
            "return(message=true, store=false)",
            ]}}
        [response, message] = _do_arax_query(query)
        print(response.show())
>       assert response.status == 'OK'
E       AssertionError: assert 'ERROR' == 'OK'
E         - OK
E         + ERROR

test_ARAX_overlay.py:158: AssertionError
---------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------
Response:
  status: ERROR
  n_errors: 1  n_warnings: 1  n_messages: 44
  error_code: UncaughtARAXiError   message: An uncaught error occurred: unable to open database file: ['Traceback (most recent call last):\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/test/../ARAXQuery/ARAX_query.py", line 501, in executeProcessingPlan\n    result = overlay.apply(message, action[\'parameters\'], response=response)\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/test/../ARAXQuery/ARAX_overlay.py", line 141, in apply\n    getattr(self, \'_\' + self.__class__.__name__ + \'__\' + parameters[\'action\'])()  # thank you https://stackoverflow.com/questions/11649848/call-methods-by-string\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/test/../ARAXQuery/ARAX_overlay.py", line 217, in __compute_ngd\n    response = NGD.compute_ngd()\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/test/../ARAXQuery/Overlay/compute_ngd.py", line 81, in compute_ngd\n    ngd_value, method_used = self.NGD.get_ngd_for_all_fast([source_curie, target_curie], [source_name, target_name])\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/NormGoogleDistance.py", line 89, in get_ngd_for_all_fast\n    return NormGoogleDistance.get_ngd_for_all(curie_id_list, description_list), "slow"\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/NormGoogleDistance.py", line 255, in get_ngd_for_all\n    terms[a] = NormGoogleDistance.get_mesh_term_for_all(curie_id_list[a], description_list[a])\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/NormGoogleDistance.py", line 169, in get_mesh_term_for_all\n    if QueryNCBIeUtils.is_mesh_term(description):\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryNCBIeUtils.py", line 501, in is_mesh_term\n    ret_list = QueryNCBIeUtils.get_mesh_uids_for_mesh_term(mesh_term)\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryNCBIeUtils.py", line 175, in get_mesh_uids_for_mesh_term\n    \'db=mesh&term=\' +  urllib.parse.quote(mesh_term + \'[MeSH Terms]\', safe=\'\'))\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryNCBIeUtils.py", line 64, in send_query_get\n    requests = CacheControlHelper()\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/cache_control_helper.py", line 32, in __init__\n    self.sess = CacheControl(requests.session(), heuristic=CustomHeuristic(days=30), cache=FileCache(\'.web_cache\'))\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests/sessions.py", line 770, in session\n    return Session()\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/core.py", line 212, in __init__\n    **backend_options\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/core.py", line 73, in __init__\n    self.cache = backends.create_backend(backend, cache_name, backend_options)\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/__init__.py", line 63, in create_backend\n    return registry[backend_name](cache_name, **options)\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/sqlite.py", line 28, in __init__\n    self.responses = DbPickleDict(location + extension, \'responses\', fast_save=fast_save)\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/storage/dbdict.py", line 62, in __init__\n    with self.connection() as con:\n', '  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 112, in __enter__\n    return next(self.gen)\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/storage/dbdict.py", line 74, in connection\n    con = sqlite.connect(self.filename)\n', 'sqlite3.OperationalError: unable to open database file\n']
  - 2020-06-11 16:07:29.902495 WARNING: Utilizing API calls to NCBI eUtils, so this may take a while...
  - 2020-06-11 16:07:29.904412 ERROR: An uncaught error occurred: unable to open database file: ['Traceback (most recent call last):\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/test/../ARAXQuery/ARAX_query.py", line 501, in executeProcessingPlan\n    result = overlay.apply(message, action[\'parameters\'], response=response)\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/test/../ARAXQuery/ARAX_overlay.py", line 141, in apply\n    getattr(self, \'_\' + self.__class__.__name__ + \'__\' + parameters[\'action\'])()  # thank you https://stackoverflow.com/questions/11649848/call-methods-by-string\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/test/../ARAXQuery/ARAX_overlay.py", line 217, in __compute_ngd\n    response = NGD.compute_ngd()\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/test/../ARAXQuery/Overlay/compute_ngd.py", line 81, in compute_ngd\n    ngd_value, method_used = self.NGD.get_ngd_for_all_fast([source_curie, target_curie], [source_name, target_name])\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/NormGoogleDistance.py", line 89, in get_ngd_for_all_fast\n    return NormGoogleDistance.get_ngd_for_all(curie_id_list, description_list), "slow"\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/NormGoogleDistance.py", line 255, in get_ngd_for_all\n    terms[a] = NormGoogleDistance.get_mesh_term_for_all(curie_id_list[a], description_list[a])\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/NormGoogleDistance.py", line 169, in get_mesh_term_for_all\n    if QueryNCBIeUtils.is_mesh_term(description):\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryNCBIeUtils.py", line 501, in is_mesh_term\n    ret_list = QueryNCBIeUtils.get_mesh_uids_for_mesh_term(mesh_term)\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryNCBIeUtils.py", line 175, in get_mesh_uids_for_mesh_term\n    \'db=mesh&term=\' +  urllib.parse.quote(mesh_term + \'[MeSH Terms]\', safe=\'\'))\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryNCBIeUtils.py", line 64, in send_query_get\n    requests = CacheControlHelper()\n', '  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/cache_control_helper.py", line 32, in __init__\n    self.sess = CacheControl(requests.session(), heuristic=CustomHeuristic(days=30), cache=FileCache(\'.web_cache\'))\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests/sessions.py", line 770, in session\n    return Session()\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/core.py", line 212, in __init__\n    **backend_options\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/core.py", line 73, in __init__\n    self.cache = backends.create_backend(backend, cache_name, backend_options)\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/__init__.py", line 63, in create_backend\n    return registry[backend_name](cache_name, **options)\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/sqlite.py", line 28, in __init__\n    self.responses = DbPickleDict(location + extension, \'responses\', fast_save=fast_save)\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/storage/dbdict.py", line 62, in __init__\n    with self.connection() as con:\n', '  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 112, in __enter__\n    return next(self.gen)\n', '  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/storage/dbdict.py", line 74, in connection\n    con = sqlite.connect(self.filename)\n', 'sqlite3.OperationalError: unable to open database file\n']

_____________________________________________________________ test_compute_ngd_attribute ______________________________________________________________

    def test_compute_ngd_attribute():
        query = {"previous_message_processing_plan": {"processing_actions": [
            "create_message",
            "add_qnode(name=DOID:384, id=n00)",
            "add_qnode(type=chemical_substance, is_set=true, id=n01)",
            "add_qedge(source_id=n00, target_id=n01, id=e00)",
            "expand(edge_id=e00)",
            "overlay(action=compute_ngd)",
            "resultify(ignore_edge_direction=true, debug=true)",
            "return(message=true, store=false)",
            ]}}
        [response, message] = _do_arax_query(query)
        print(response.show())
>       assert response.status == 'OK'
E       AssertionError: assert 'ERROR' == 'OK'
E         - OK
E         + ERROR

test_ARAX_overlay.py:183: AssertionError
---------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------
Response:
  status: ERROR
  n_errors: 2  n_warnings: 1  n_messages: 52
  error_code: UnknownError   message: Something went wrong adding the NGD edge attributes
  - 2020-06-11 16:07:30.632937 WARNING: Utilizing API calls to NCBI eUtils, so this may take a while...
  - 2020-06-11 16:07:30.633587 ERROR: Traceback (most recent call last):
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/test/../ARAXQuery/Overlay/compute_ngd.py", line 134, in compute_ngd
    ngd_value, method_used = self.NGD.get_ngd_for_all_fast([source_curie, target_curie], [source_name, target_name])
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/NormGoogleDistance.py", line 89, in get_ngd_for_all_fast
    return NormGoogleDistance.get_ngd_for_all(curie_id_list, description_list), "slow"
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/NormGoogleDistance.py", line 255, in get_ngd_for_all
    terms[a] = NormGoogleDistance.get_mesh_term_for_all(curie_id_list[a], description_list[a])
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/NormGoogleDistance.py", line 169, in get_mesh_term_for_all
    if QueryNCBIeUtils.is_mesh_term(description):
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryNCBIeUtils.py", line 501, in is_mesh_term
    ret_list = QueryNCBIeUtils.get_mesh_uids_for_mesh_term(mesh_term)
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryNCBIeUtils.py", line 175, in get_mesh_uids_for_mesh_term
    'db=mesh&term=' +  urllib.parse.quote(mesh_term + '[MeSH Terms]', safe=''))
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryNCBIeUtils.py", line 64, in send_query_get
    requests = CacheControlHelper()
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/cache_control_helper.py", line 32, in __init__
    self.sess = CacheControl(requests.session(), heuristic=CustomHeuristic(days=30), cache=FileCache('.web_cache'))
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests/sessions.py", line 770, in session
    return Session()
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/core.py", line 212, in __init__
    **backend_options
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/core.py", line 73, in __init__
    self.cache = backends.create_backend(backend, cache_name, backend_options)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/__init__.py", line 63, in create_backend
    return registry[backend_name](cache_name, **options)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/sqlite.py", line 28, in __init__
    self.responses = DbPickleDict(location + extension, 'responses', fast_save=fast_save)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/storage/dbdict.py", line 62, in __init__
    with self.connection() as con:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/storage/dbdict.py", line 74, in connection
    con = sqlite.connect(self.filename)
sqlite3.OperationalError: unable to open database file

  - 2020-06-11 16:07:30.633603 ERROR: Something went wrong adding the NGD edge attributes

________________________________________________________ test_paired_concept_frequency_virtual ________________________________________________________

    def test_paired_concept_frequency_virtual():
        query = {"previous_message_processing_plan": {"processing_actions": [
            "create_message",
            "add_qnode(curie=DOID:1588, id=n0)",
            "add_qnode(type=chemical_substance, id=n1)",
            "add_qedge(source_id=n0, target_id=n1, id=e0)",
            "expand(edge_id=e0)",
            "overlay(action=overlay_clinical_info, paired_concept_frequency=true, source_qnode_id=n0, target_qnode_id=n1, virtual_relation_label=CP1)",
            "resultify()",
            "return(message=true, store=false)",
        ]}}
        [response, message] = _do_arax_query(query)
        print(response.show())
>       assert response.status == 'OK'
E       AssertionError: assert 'ERROR' == 'OK'
E         - OK
E         + ERROR

test_ARAX_overlay.py:236: AssertionError
---------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------
Response:
  status: ERROR
  n_errors: 332  n_warnings: 0  n_messages: 381
  error_code: UnknownError   message: Something went wrong when adding the edge attribute from COHD.
  - 2020-06-11 16:09:42.470304 ERROR: Traceback (most recent call last):
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/test/../ARAXQuery/Overlay/overlay_clinical_info.py", line 114, in make_edge_attribute_from_curies
    source_OMOPs = [str(x['omop_standard_concept_id']) for x in COHD.get_xref_to_OMOP(source_curie, 1)]
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryCOHD.py", line 384, in get_xref_to_OMOP
    res_json = QueryCOHD.__access_api(handler, url_suffix)
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryCOHD.py", line 61, in __access_api
    requests = CacheControlHelper()
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/cache_control_helper.py", line 32, in __init__
    self.sess = CacheControl(requests.session(), heuristic=CustomHeuristic(days=30), cache=FileCache('.web_cache'))
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests/sessions.py", line 770, in session
    return Session()
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/core.py", line 212, in __init__
    **backend_options
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/core.py", line 73, in __init__
    self.cache = backends.create_backend(backend, cache_name, backend_options)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/__init__.py", line 63, in create_backend
    return registry[backend_name](cache_name, **options)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/sqlite.py", line 28, in __init__
    self.responses = DbPickleDict(location + extension, 'responses', fast_save=fast_save)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/storage/dbdict.py", line 62, in __init__
    with self.connection() as con:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/storage/dbdict.py", line 74, in connection
    con = sqlite.connect(self.filename)
sqlite3.OperationalError: unable to open database file

  - 2020-06-11 16:09:42.470332 ERROR: Something went wrong when adding the edge attribute from COHD.
  - 2020-06-11 16:09:42.470866 ERROR: Traceback (most recent call last):
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/test/../ARAXQuery/Overlay/overlay_clinical_info.py", line 114, in make_edge_attribute_from_curies
    source_OMOPs = [str(x['omop_standard_concept_id']) for x in COHD.get_xref_to_OMOP(source_curie, 1)]
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryCOHD.py", line 384, in get_xref_to_OMOP
    res_json = QueryCOHD.__access_api(handler, url_suffix)
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/QueryCOHD.py", line 61, in __access_api
    requests = CacheControlHelper()
  File "/Users/aglen/translator/RTX/RTX/code/ARAX/ARAXQuery/../../reasoningtool/kg-construction/cache_control_helper.py", line 32, in __init__
    self.sess = CacheControl(requests.session(), heuristic=CustomHeuristic(days=30), cache=FileCache('.web_cache'))
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests/sessions.py", line 770, in session
    return Session()
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/core.py", line 212, in __init__
    **backend_options
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/core.py", line 73, in __init__
    self.cache = backends.create_backend(backend, cache_name, backend_options)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/__init__.py", line 63, in create_backend
    return registry[backend_name](cache_name, **options)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/sqlite.py", line 28, in __init__
    self.responses = DbPickleDict(location + extension, 'responses', fast_save=fast_save)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/storage/dbdict.py", line 62, in __init__
    with self.connection() as con:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/Users/aglen/translator/RTX/rtx-env-7/lib/python3.7/site-packages/requests_cache/backends/storage/dbdict.py", line 74, in connection
    con = sqlite.connect(self.filename)
sqlite3.OperationalError: unable to open database file

[edeutsch]

I am testing on main server. But is taking a very long time. presumably populating NGD cache. 35% done all PASSED so far.

[edeutsch]

All tests passed on the main server. But there were a bunch of warnings about using deprecated modules:

test_ARAX_overlay.py::test_predict_drug_treats_disease_virtual /mnt/data/python/Python-3.7.3/lib/python3.7/site-packages/sklearn/externals/joblib/init.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+. warnings.warn(msg, category=FutureWarning)

test_ARAX_overlay.py::test_predict_drug_treats_disease_virtual /mnt/data/python/Python-3.7.3/lib/python3.7/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.linear_model.logistic module is deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.linear_model. Anything that cannot be imported from sklearn.linear_model is now part of the private API. warnings.warn(message, FutureWarning)

test_ARAX_overlay.py::test_predict_drug_treats_disease_virtual test_ARAX_overlay.py::test_predict_drug_treats_disease_attribute test_ARAX_resultify.py::TestARAXResultify::test_example2 test_ARAX_resultify.py::TestARAXResultify::test_issue680 test_ARAX_workflows.py::test_example_2 /mnt/data/python/Python-3.7.3/lib/python3.7/site-packages/sklearn/base.py:318: UserWarning: Trying to unpickle estimator LogisticRegression from version 0.19.1 when using version 0.22.2.post1. This might lead to breaking code or invalid results. Use at your own risk. UserWarning)

[amykglen]

ok, great, thanks @edeutsch. well I suppose I'll finish up the merge then and merge issue720 into master since apparently the failures are just a problem in my local environment.

[amykglen]

Merge is complete! No conflicts.

Now I'm just going to merge the expander and resultify branches into master for #722 and #800 - will let you know when that is done and we're ready for a roll-out, @edeutsch.