Add Counts from Implicit Edge Creation to Expansion Menus

[everbeek]

As relations come in, increment a per-node counter, and update the expansion menu immediately (even if it is currently open). Look in manifestOrRegisterImplicitRelation(). Alternately, increment in expandAndParseNodeIfNeeded().

[everbeek]

Doing this showed me a better way of deactivating the expansion menus, by simply looking at the edge registry and checking for null endpoints. I am also noticing inconsistencies in the expansion numbers. For example, ligamentum arteriosum says it has 7 to expand, and when I do, I have only 5 arcs. Looking for the explanation...

1) It has a double arc with one concept, and this arc disappears when the menu is activated, and made visible when the node is moved (ductus arteriosus, "develops_from" and "treeView" 2) Two arcs are "database_cross_reference", which are not concept relations.

As part of this issue, because it facilitates the work, I need to figure out if I can identify concept relations (didn't find a way before), or hard-code ones like the databsae_cross_reference so that they are not entered as edges into the edge registry.

[everbeek]

I can exclude any property that happens to have even a single non-URL value, but that is no guarantee, so I don't know if I should sweep things under the rug by doing that.

[everbeek]

Ok, how about this...relation properties that are within the same ontology are clearly identifiable. They have URL values that have the same prefix, though the structure of each prefix can differ.

I need to see a single concept from every single ontology so I can know if this approach will work. Generating that data from the ontology overview, since things are close to that already there.

There are a few formats for concept urls: -purl ones are all similar, and the prefix includes the ontology acronym (http://purl.bioontology.org/ontology/ICD10/P00-P96.9) -prefix ending at # (http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#C17828, or http://phenomebrowser.net/ontologies/mesh/mesh.owl#B02) -sometimes the purl ones have a different ontology identifier than the ontology acronym, but they still go ontology/[^//]/.+

I have found the BIOMODELS ontology, which uses concepts from other ontologies. This means that no two related concepts necessarily share an id prefix. It uses two, a purl one and an owl one (with # separating the concept out). Depending on how the rest turn out, perhaps I can check against either format, rather than comparing a candidate url to the parent related url...

FMA has diverse urls, without the .owl#, but with fma# and rdf-schema#. I am having more doubts...

[everbeek]

I think I have to take another approach, and remove edges for which some endpoint nodes can not be retrieved. The trouble being...403 errors! That is the only way to detect urls that do not correspond to concept ids. If I am right, then I need CORS working before I can get the edge counts sorted out.

Then again, I could polish the expansion set check that was in place, and orient it towards whether the request was made for each concept, rather than if it is actually in the graph. if the number available compared to the number requested match, then the expansion is done. When giving numbers for the number of concepts in an expansion, they will be overestimates sometimes.

[everbeek]

No, it isn't working out to track REST requests.

I am fairly sure the only way to do it is to switch to using CORS so I can receive error codes, then to remove erroneous nodes from the graph's edge registry. Then, when I compare the total known edges of a concept to the edges extant, there will not be this permanent difference of the number of edges that were not valid concepts or that are not available due to permissions. This doesn't rely on expansions sets in a way that would get broken by undo/redo, because we always have the complete set of potential edges, and we have the set of extant edges at the current step...removing the invalid ones from the potential edges brings things in line over time. There is always the chance that we think there are more than there really are when we have not tried to fetch all of them, but that merely inflates the UI count prior to expanding.

If I try to track fetch requests, undo/redo problems occur; I would need to count all requests in all expansion sets that have the node in question as their parent, and the code for when to register a request is inconvenient (at parseNode() for children, parents, but at fetch() calls for composition relations and mappings).

I need the error codes for other things, so I am going to tidy up what I have, stash it, then get on CORS, and come back.

[everbeek]

I hit an issue with CORS, and decided to give this another shot without it.

I tried tracking fetch calls in the expansions sets in a different way, and it brought me closer to what I envisioned before, but...if any nodes were deleted since the expansion in question, they are not accounted for. This is the fourth time I have tried to cope with erroneous nodes, without using error codes from CORS, but I cannot.

If I had CORS, would I actually be able to deal with deletions? Yes, because I could very simply compare the extant nodes to those the expansion set knows about. Without the ability ot handle errors, erroneous nodes would perpetually look like they had not been expanded due to capping, or having been deleted, both of which are cases where we do want to be able to expand again.

Stashed this again, for when I do have CORS working.

$ git stash save "issue 447, failed at making expansion node counts work without error handling, which requir es CORS fetches. Pick up from this when I have CORS." Saved working directory and index state On master: issue 447, failed at making expansion node counts work with out error handling, which requires CORS fetches. Pick up from this when I have CORS.

[everbeek]

Have things sorted out finally, but I have to clean up some really crufty previous attempts. I also need to get it working for the ontology overview while I'm in here...but perhaps I could commit the clean copy first.

In particular, I have to decide whether to remove some facilities I added to expansion sets, or correct them to work properly while breaking encapsulation. I question the future helpfulness of the numTotal, numMissing, numAccountedFor properties, and the associated methods used to calculate those.

[everbeek]

Now, I need to see if the ontology overview can benefit from error handling. This cna be a new case, because I haven't had issues like that yet. See #482.