HazyResearch / deepdive

DeepDive
deepdive.stanford.edu
1.96k stars 539 forks source link

Wrong grounding ID assignment? #56

Closed zifeishan closed 10 years ago

zifeishan commented 10 years ago

I'm running on madmax (with GP database) and sampling failed. it seems that variable IDs are not correct (0--144847, while count is only 75454).

While I am running on Mac (with PSQL) it seems fine.

Logs:

01:58:20 [PostgresInferenceDataStoreComponent$PostgresInferenceDataStore(akka://deepdive)] INFO  Dumping factor graph...
01:58:20 [PostgresInferenceDataStoreComponent$PostgresInferenceDataStore(akka://deepdive)] INFO  Dumping variables...
01:58:20 [] DEBUG Borrowed a new connection from ConnectionPool(url:jdbc:postgresql://rambo:5433/deepdive_spouse_plpy, user:zifei)
01:58:21 [PostgresInferenceDataStoreComponent$PostgresInferenceDataStore(akka://deepdive)] INFO  Dumping weights...
01:58:21 [] DEBUG Borrowed a new connection from ConnectionPool(url:jdbc:postgresql://rambo:5433/deepdive_spouse_plpy, user:zifei)
01:58:21 [PostgresInferenceDataStoreComponent$PostgresInferenceDataStore(akka://deepdive)] INFO  Dumping factors...
01:58:21 [] DEBUG Borrowed a new connection from ConnectionPool(url:jdbc:postgresql://rambo:5433/deepdive_spouse_plpy, user:zifei)
01:58:21 [] DEBUG Borrowed a new connection from ConnectionPool(url:jdbc:postgresql://rambo:5433/deepdive_spouse_plpy, user:zifei)
01:58:22 [sampler] INFO  starting
01:58:22 [sampler] INFO  util/sampler-dw-linux gibbs
01:58:22 [sampler] INFO  Executing: util/sampler-dw-linux gibbs -w /lfs/madmax/0/zifei/deepdive/out/2014-05-03T015805/graph.weights -v /lfs/madmax/0/zifei/deepdive/out/2014-05-03T015805/graph.variables -f /lfs/madmax/0/zifei/deepdive/out/2014-05-03T015805/graph.factors -e /lfs/madmax/0/zifei/deepdive/out/2014-05-03T015805/graph.edges -m /lfs/madmax/0/zifei/deepdive/out/2014-05-03T015805/graph.meta.csv -o /lfs/madmax/0/zifei/deepdive/out/2014-05-03T015805 -l 150 -s 1 -i 200 --alpha 0.1
01:58:22 [sampler] INFO
01:58:22 [sampler] INFO  #################MACHINE CONFIG#################
01:58:22 [sampler] INFO  # # NUMA Node        : 8
01:58:22 [sampler] INFO  # # Thread/NUMA Node : 8
01:58:22 [sampler] INFO  ################################################
01:58:22 [sampler] INFO
01:58:22 [sampler] INFO  #################GIBBS SAMPLING#################
01:58:22 [sampler] INFO  # fg_file            : /lfs/madmax/0/zifei/deepdive/out/2014-05-03T015805/graph.meta.csv
01:58:22 [sampler] INFO  # edge_file          : /lfs/madmax/0/zifei/deepdive/out/2014-05-03T015805/graph.edges
01:58:22 [sampler] INFO  # weight_file        : /lfs/madmax/0/zifei/deepdive/out/2014-05-03T015805/graph.weights
01:58:22 [sampler] INFO  # variable_file      : /lfs/madmax/0/zifei/deepdive/out/2014-05-03T015805/graph.variables
01:58:22 [sampler] INFO  # factor_file        : /lfs/madmax/0/zifei/deepdive/out/2014-05-03T015805/graph.factors
01:58:22 [sampler] INFO  # output_folder      : /lfs/madmax/0/zifei/deepdive/out/2014-05-03T015805
01:58:22 [sampler] INFO  # n_learning_epoch   : 150
01:58:22 [sampler] INFO  # n_samples/l. epoch : 1
01:58:22 [sampler] INFO  # n_inference_epoch  : 200
01:58:22 [sampler] INFO  # stepsize           : 0.1
01:58:22 [sampler] INFO  # decay              : 0.95
01:58:22 [sampler] INFO  ################################################
01:58:22 [sampler] INFO  # IGNORE -s (n_samples/l. epoch). ALWAYS -s 1. #
01:58:22 [sampler] INFO  # IGNORE -t (threads). ALWAYS USE ALL THREADS. #
01:58:22 [sampler] INFO  ################################################
01:58:22 [sampler] INFO  # nvar               : 75454
01:58:22 [sampler] INFO  # nfac               : 154835
01:58:22 [sampler] INFO  # nweight            : 23449
01:58:22 [sampler] INFO  # nedge              : 230289
01:58:22 [sampler] INFO  ################################################
01:58:22 [sampler] INFO  LOADED VARIABLES: #75454
01:58:22 [sampler] INFO           N_QUERY: #71555
01:58:22 [sampler] INFO           N_EVID : #3899
01:58:22 [sampler] INFO  LOADED FACTORS: #154835
01:58:22 [sampler] INFO  LOADED WEIGHTS: #23449
01:58:22 [inferenceManager] ERROR sampling failed (see error log for more details)
java.lang.RuntimeException: sampling failed (see error log for more details)
    at org.deepdive.inference.Sampler$$anonfun$receive$1.applyOrElse(Sampler.scala:36) ~[classes/:na]
    at akka.actor.Actor$class.aroundReceive(Actor.scala:467) ~[akka-actor_2.10-2.3-M2.jar:2.3-M2]
    at org.deepdive.inference.Sampler.aroundReceive(Sampler.scala:17) ~[classes/:na]
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:491) ~[akka-actor_2.10-2.3-M2.jar:2.3-M2]
    at akka.actor.ActorCell.invoke(ActorCell.scala:462) ~[akka-actor_2.10-2.3-M2.jar:2.3-M2]
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237) ~[akka-actor_2.10-2.3-M2.jar:2.3-M2]
    at akka.dispatch.Mailbox.run(Mailbox.scala:219) ~[akka-actor_2.10-2.3-M2.jar:2.3-M2]
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:385) ~[akka-actor_2.10-2.3-M2.jar:2.3-M2]
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) ~[scala-library.jar:na]
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) ~[scala-library.jar:na]
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) ~[scala-library.jar:na]
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) ~[scala-library.jar:na]
01:58:22 [sampler] INFO  Message [akka.actor.PoisonPill$] from Actor[akka://deepdive/user/inferenceManager#1673605566] to Actor[akka://deepdive/user/inferenceManager/sampler#-1740430581] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.

FYI, var ID min/max/count:

deepdive_spouse_plpy=# select min(id) from has_spouse;

min

0 (1 row)

deepdive_spouse_plpy=# select max(id) from has_spouse;

max

144847 (1 row)

deepdive_spouse_plpy=# select count(distinct id) from has_spouse;

count

75454 (1 row)

feiranwang commented 10 years ago

needs full log to see what happened...

feiranwang commented 10 years ago

Fixed in f7abdd0b2da84f2122f191e9ca71021cecb55472