bio4j / bio4j-titan

Titan-specific bio4j implementation
https://github.com/bio4j/bio4j
6 stars 2 forks source link

The new version takes forever!! #49

Closed pablopareja closed 9 years ago

pablopareja commented 9 years ago

I'm performing the first tests to import Bio4j using titan 0.5.1 and at least as it is right now just takes forever to import GO ! :fearful: :cold_sweat:

Importing Gene Ontology was taking around 1 minute with the previous version and surprisingly would take now ~ 1 hour !

Any ideas why?

I assume we're not properly setting up some configuration parameters because otherwise this is slow as hell....! :fire:

eparejatobes commented 9 years ago

no idea :-|

eparejatobes commented 9 years ago

@pablopareja did you run the EnzymeDB test?

eparejatobes commented 9 years ago

I am looking at this

pablopareja commented 9 years ago

yeah, the thing is that in the case of EnzymeDB there's no index look up in the whole process so the time spent is equivalent to that when using the older version

eparejatobes commented 9 years ago

It looks like this is scanning the whole thing right now, indexes are not being picked up. Let me check that this is exactly what's going on.

eparejatobes commented 9 years ago

Yes, that's it. Let's see what else do we need here

eparejatobes commented 9 years ago

I mean for indexes being used.

eparejatobes commented 9 years ago

@pablopareja fixed! I'll push in a couple of minutes. Anyway labels and indexes are pretty scary, read this:

eparejatobes commented 9 years ago
[info] GOTestSuite:
[info] - Importing GO
[info] Run completed in 1 minute, 35 seconds.
eparejatobes commented 9 years ago

!!!

pablopareja commented 9 years ago

cool! :smiley:

So they're becoming Neo4j-ish and Neo4j is sort of going into the right direction ?? things are weird... :confused:

eparejatobes commented 9 years ago

@pablopareja btw this was slower before right? I mean GO took several minutes with the previous version IIRC

pablopareja commented 9 years ago

Not really: 1m 14s

eparejatobes commented 9 years ago

OK fine, then things are at least the same as before

eparejatobes commented 9 years ago

closing, will update to the new angulillos in a diff branch