CostFed times out randomly

[mhoangvslev]

• We take 4 measurements for the same query instance, some of which occasionally time out.

[mhoangvslev]

• Executing the java command gives this error message:

  00:20:30.267 [Evaluation Scheduler_1] INFO  o.a.s.q.core.evaluation.HashJoinImpl - 
  00:20:30.267 [Evaluation Scheduler_0] INFO  o.a.s.q.core.evaluation.HashJoinImpl - 
  Exception in thread "Evaluation Scheduler_0" java.lang.NullPointerException
      at org.aksw.simba.quetsal.core.evaluation.HashJoinImpl.handleClose(HashJoinImpl.java:254)
      at com.fluidops.fedx.evaluation.iterator.RestartableCloseableIteration.close(RestartableCloseableIteration.java:34)
      at com.fluidops.fedx.evaluation.union.UnionExecutorBase$1.release(UnionExecutorBase.java:63)
      at com.fluidops.fedx.evaluation.union.UnionExecutorBase$1.release(UnionExecutorBase.java:60)
      at com.fluidops.fedx.evaluation.iterator.QueueIterator.add_release(QueueIterator.java:77)
      at com.fluidops.fedx.evaluation.union.UnionExecutorBase$UnionTask.callAsync(UnionExecutorBase.java:92)
      at com.fluidops.fedx.evaluation.union.UnionExecutorBase$UnionTask.callAsync(UnionExecutorBase.java:74)
      at com.fluidops.fedx.evaluation.concurrent.Async.run(Async.java:25)
      at com.fluidops.fedx.evaluation.concurrent.ControlledWorkerScheduler.threadProc(ControlledWorkerScheduler.java:106)
      at com.fluidops.fedx.evaluation.concurrent.ControlledWorkerScheduler.access$000(ControlledWorkerScheduler.java:21)
      at com.fluidops.fedx.evaluation.concurrent.ControlledWorkerScheduler$1.run(ControlledWorkerScheduler.java:50)
      at java.base/java.lang.Thread.run(Thread.java:829)

• This error is already reported without follow-ups from the original authors.

[mhoangvslev]

• Another instance of error that leads to timeout where everything is done but it get stuck here.

01:03:07.904 [main] INFO  com.fluidops.fedx.FedX - Shutting down federation and all underlying repositories ...

org.eclipse.rdf4j.query.QueryEvaluationException: java.lang.InterruptedException: cancelled
        at com.fluidops.fedx.evaluation.iterator.QueueIterator.checkException(QueueIterator.java:111)
        at com.fluidops.fedx.evaluation.iterator.QueueIterator.getNextElement(QueueIterator.java:41)
        at com.fluidops.fedx.evaluation.iterator.RestartableLookAheadIteration.lookAhead(RestartableLookAheadIteration.java:50)
        at com.fluidops.fedx.evaluation.iterator.RestartableLookAheadIteration.hasNext(RestartableLookAheadIteration.java:21)
        at com.fluidops.fedx.evaluation.union.UnionExecutorBase.getNextElement(UnionExecutorBase.java:111)
        at org.eclipse.rdf4j.common.iteration.LookAheadIteration.lookAhead(LookAheadIteration.java:90)
        at org.eclipse.rdf4j.common.iteration.LookAheadIteration.hasNext(LookAheadIteration.java:52)
        at com.fluidops.fedx.evaluation.iterator.QueueIteration.getNextElement(QueueIteration.java:44)

[hartig]

I think it is not our job to fix bugs in CostFed, not even to identify the cause of them.

I suggest if less than half of the executions of a query time out, you ignore the timed-out ones and consider only the non-timeout measurements. Otherwise, you count the query as timed out or as failed.

Additionally, if you want to be nice, you post a bug report at the CostFed repo, but I wouldn't overdo it in terms of putting a lot of time into it.