search

protegeproject / webprotege

The webprotege code base
Other
623 stars 249 forks source link

Some large ontologies (NCBI Taxon for example) take over a minute to load #635

Open matthewhorridge opened 5 years ago

matthewhorridge commented 5 years ago

From the mailing list:

Hi all, I am trying to get a local instance of web protégé configured to deal with our ontologies.

I am doing several tests with different ontologies and I am getting some performance issues. For example, I uploaded NCBI Taxon ontology (1.35GB) into web protégé and, every time I want to open it, it takes more than one minute. I am using web protégé 4.0.0 beta-2, and tomcat has 20G of memory assigned. I think setting the dormant project time property to prevent WebProtege from ejecting ontologies from memory is not a feasible option because users are going to upload more ontologies when the system goes to production environment, and I am not very sure if all of them would fit in memory.

Do you have any tip to improve the web protégé loading performance with big ontologies?

matthewhorridge commented 5 years ago

@fanavarro I've transferred this issue from the mailing list to here.

While I don't currently have any tips to improve the loading performance of ontologies, I do have a question for you: In your production environment, are all the ontologies very large, like NCBI Taxon or are the likely to be smaller? Much smaller?

fanavarro commented 5 years ago

Hi @matthewhorridge.

A great part of our ontologies are smaller. However, we have a couple of them that are larger than NCBI Taxon.

matthewhorridge commented 5 years ago

@fanavarro the reason for me asking is that I was wondering whether some more fine-grained dormant project time could be necessary. Small ontologies could have short default purge times and larger ontologies could have specific purge times set for them.

I've been doing some profiling and there's some improvements that I can make. Load time would be 2-3 times quicker, not an order of magnitude quicker, but the improvements are worth making. They require a fair bit of work though, so not something that could be released in the next few days.

fanavarro commented 5 years ago

Thanks for your answer @matthewhorridge. What you said is very interesting, it makes sense that purge time should be dependent on load time. I will check our settings.

Having a load time 2-3 times quicker would be fantastic, but I know this kind of improvement is difficult to achieve.

Thanks for your answer again!