search

scipion-em / scipion-em

Base Scipion plugin defining objects and protocols for EM
GNU General Public License v3.0
8 stars 5 forks source link

subset selection performance #187

Open azazellochg opened 3 years ago

azazellochg commented 3 years ago

I guess I'll make an issue so we can start digging into this long standing problem :)

I'm doing now a simplest case - selecting particles from ONE 3D class (out of 3). Total number of particles is 138436 - not much at all After clicking OK to create a subset, the whole thing (except Project window) is frozen. Already 3h past it is still populating classes3D_state.sqlite file. It has not even created the subset protocol yet

azazellochg commented 3 years ago

ok, I think I killed the performance with my own recent plugin changes (not released). Will have to review. It's never been so long..

pconesa commented 3 years ago

are you using "performance" branch?

pconesa commented 3 years ago

For what you are saying that is happening in Java code? (ShowJ)

azazellochg commented 3 years ago

I dont use performance branch. Yes, it's java code running forever. Create subset protocol takes "only" 1 hour. In my plugin I have added more columns into particles table (63 columns now) so I think this is killing the performance

pconesa commented 3 years ago

Yes, the more columns the more we will struggle.

pconesa commented 3 years ago

Hi, @azazellochg , do you need those 63 fields? Issues related to Java code are harder to address. I don't think we can address this anytime soon unless is python related code.

pconesa commented 3 years ago

There is a team in Czech republic addressing performance issues in the binding and metadata handling... this is the better chance we have for this.