Issue either with models not ingested in blazegraph after a relaunch OR minerva Search API

[lpalbou]

We were discussing Karen issue with title search: https://github.com/geneontology/noctua-annotation-review/issues/44 .

For reference, there are indeed two models with the PMID 27340223, both in the title and in the evidence fields:

• https://github.com/geneontology/noctua-models/blob/dev/models/59c8885900000227.ttl
• https://github.com/geneontology/noctua-models/blob/dev/models/5900dc7400000468.ttl

Searching ty title query: http://barista-dev.berkeleybop.org/search//models?offset=0&limit=50&title=27340223&expand&debug Searching by mid query: http://barista-dev.berkeleybop.org/search//models?offset=0&limit=50&pmid=PMID:27340223&expand&debug

Only the 59c8885900000227 model can be found, not the 5900dc7400000468.

@tmushayahama had the idea it could come from models not being ingested back at minerva/blazegraph launch, and at first I agreed, but then I tested on what I believe is the RDF/SPARQL for dev: http://rdf-internal.berkeleybop.io/blazegraph/#query and I was able to find both models with a simple query:

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> 
PREFIX metago: <http://model.geneontology.org/>

SELECT ?gocam  WHERE {
  GRAPH ?gocam {
    ?gocam metago:graphType metago:noctuaCam .
    ?gocam dc:title "PMID:27340223 Dnah11 Mks1 - KRC"^^<http://www.w3.org/2001/XMLSchema#string> ;
  }
}

=> does find http://model.geneontology.org/59c8885900000227

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> 
PREFIX metago: <http://model.geneontology.org/>

SELECT ?gocam  WHERE {
  GRAPH ?gocam {
    ?gocam metago:graphType metago:noctuaCam .
    ?gocam dc:title "PMID-27340223-krc"^^<http://www.w3.org/2001/XMLSchema#string>
  }
}

=> does find http://model.geneontology.org/5900dc7400000468

IF, http://rdf-internal.berkeleybop.io/blazegraph/#query is indeed the up to date RDF/SPARQL for dev, then the error is not in Minerva/blazegraph not ingesting models... IF it's not, bets are open, and it's either that or the Minerva API not working.

Action items:

• check directly with blazegraph if those models are ingested
• create a test @minerva/blazegraph launch that the number of models ingested = number of models in GH (TTL)
• fix either minerva/blazegraph ingest or minerva API

@cmungall @kltm @balhoff as this may be a show stopper issue if that's in blazegraph ingest

[balhoff]

https://github.com/geneontology/noctua-models/blob/dev/models/5900dc7400000468.ttl has modelState "delete"—it will not be loaded by Minerva at startup.

[lpalbou]

Great catch @balhoff !

Closing that ticket.

[lpalbou]

As a note however, it would still be nice to have a test during minerva/blazegraph launch: nb models without modelState "delete" == nb models available in blazegraph

[tmushayahama]

@balhoff @lpalbou why are the models with delete modelstate not loaded at startup? We used to filter by delete modelstate. I thought they are loaded, but state can be changed i.e. from Delete back to Dev/Production. I think this would be great while waiting for a proper delete functionality on models

[balhoff]

@tmushayahama I don't know. Ben added this feature but I'm not sure how the decision was made. It is easy to turn off if a different policy is decided.

[lpalbou]

Honestly I don't care much as I wouldn't even keep those models aside from the general GH versioning. They have been deleted for a reason and people are usually very conservative about that.

In a normal production environment I wouldn't let people search for models that have been deleted but I would make sure all the confirmation messages are here to avoid a deletion by mistake. In any case I support the fact they are not loaded nor searchable. There is a bunch of test models we simply don't even want to see anymore.