search

lambdamusic / Ontospy

Python library and command-line interface for inspecting and visualizing RDF models aka ontologies.
http://lambdamusic.github.io/Ontospy/
MIT License
218 stars 52 forks source link

Any idea how to improve the performance when handling large ontologies? #44

Open leonqli opened 6 years ago

lambdamusic commented 5 years ago

Do you have any sample ontology in mind? I'm trying to wrap my head around this problem so it'd be useful to have some sample models for testing.

leonqli commented 5 years ago

you may try with this ontology: http://bioportal.bioontology.org/ontologies/NCBITAXON

lambdamusic commented 5 years ago

The main problem is that ontospy attempts to build the entire ontology model in memory, and that takes time if there are many classes and properties to correlated.

I've tried using threads, but with no real performance improvements as the main tasks (extract classes, properties, concepts etc..) tend to be reliant on each other.

For very large ontologies maybe it's more indicated to use a triplestore. Otherwise I'm kind of out of ideas here..

leonqli commented 5 years ago

You may want to take look of https://pythonhosted.org/Owlready2/ It seems to having better performance on large ontologies.

lambdamusic commented 5 years ago

Thanks! Looks like they use an ad-hoc back end, maybe that's it. Will look more into it Update: the back end is an optimized SQLite index eg view here

leonqli commented 5 years ago

Yes, they use SQLite as backend. Do you think it is helpful for improving the performance of ontospy?

jclerman commented 2 years ago

It's also not too difficult to load an ontology into Apache Fuseki Jena. The main issue is the non-Python dependency (Fuseki), but once the store is running it's easy to use rdflib to mediate querying.