search

clulab / eidos

Machine reading system for World Modelers
Apache License 2.0
36 stars 24 forks source link

remove sentence classifier? #1030

Closed BeckySharp closed 3 years ago

BeckySharp commented 3 years ago

Hey all -- this takes forever on a fresh build:

18:20:26.690 [scala-execution-context-global-173] INFO  o.c.wm.eidoscommon.utils.Sourcer$ - Sourcing resource file:/Users/bsharp/github/eidos/target/bg-jobs/sbt_98994499/job-1/target/918c5ffc/593838b3/eidos_2.12-1.2.1-
SNAPSHOT.jar!/org/clulab/wm/eidos/english/sentenceClassifier/sentence_classifier_idf_weight_1000_20000.txt

Do we really need this dep still? AFAIK it wasn't shown to be super useful? Or am I mistaken?

kwalcock commented 3 years ago

I timed the loading of the component and get less than 300ms. It looks like that component is one of the last to start loading and therefore prints its message very late. However, there is still lots being done, in overlapping threads, including the one listed next in the code, which is the processorsPrimer. That runs a sentence through processors so that its lazy values and companion objects get initialized and takes about 1000 times longer. I suspect that is what is taking up so much time and is perhaps being attributed to the other component. This shouldn't have anything to do with a fresh build, like after sbt clean. Maybe a fresh run?

BeckySharp commented 3 years ago

OK thanks!

On Tue, Jun 8, 2021 at 9:20 AM Keith Alcock @.***> wrote:

External Email

I timed the loading of the component and get less than 300ms. It looks like that component is one of the last to start loading and therefore prints its message very late. However, there is still lots being done, in overlapping threads, including the one listed next in the code, which is the processorsPrimer. That runs a sentence through processors so that its lazy values and companion objects get initialized and takes about 1000 times longer. I suspect that is what is taking up so much time and is perhaps being attributed to the other component. This shouldn't have anything to do with a fresh build, like after sbt clean. Maybe a fresh run?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.