search

vitrivr / cineast

Cineast is a multi-feature content-based mulitmedia retrieval engine. It is capable of retrieving images, audio- and video sequences as well as 3d models based on edge or color sketches, textual descriptions and example objects.
MIT License
57 stars 50 forks source link

TFInvalidArgumentException for CLIP #330

Closed silvanheller closed 2 years ago

silvanheller commented 2 years ago

When running a significant amount of queries on Cineast, you start running into CLIP issues (see error log below). I have not yet found a way to trigger this error deterministically (the same query will trigger it sometimes and other times not), and it is sometimes solved after a clean & rebuild.

I'm filing this here for future reference.

2022-07-26 15:13:35.515 [ssq-0] WARN o.v.c.s.r.ContinuousQueryDispatcher - java.util.concurrent.ExecutionException: org.tensorflow.exceptions.TFInvalidArgumentException: assertion failed: [Gather indices are out of bounds, please double check the indices and retry.] [Condition x == y did not hold element-wise:] [x (LogicalAnd:0) = ] [[0 1 1...]...] [y (assert_equal_1/y:0) = ] [1]
         [[{{function_node assert_equal_1_Assert_AssertGuard_false_274}}{{node assert_equal_1/Assert/AssertGuard/Assert}}]]
        at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
        at org.vitrivr.cineast.standalone.runtime.ContinuousQueryDispatcher.extractResults(ContinuousQueryDispatcher.java:188)
        at org.vitrivr.cineast.standalone.runtime.ContinuousQueryDispatcher.doRetrieve(ContinuousQueryDispatcher.java:157)
        at org.vitrivr.cineast.standalone.runtime.ContinuousQueryDispatcher.retrieve(ContinuousQueryDispatcher.java:98)
        at org.vitrivr.cineast.standalone.util.ContinuousRetrievalLogic.retrieve(ContinuousRetrievalLogic.java:40)
        at org.vitrivr.cineast.api.websocket.handlers.queries.TemporalQueryMessageHandler.lambda$execute$9(TemporalQueryMessageHandler.java:120)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.tensorflow.exceptions.TFInvalidArgumentException: assertion failed: [Gather indices are out of bounds, please double check the indices and retry.] [Condition x == y did not hold element-wise:] [x (LogicalAnd:0) = ] [[0 1 1...]...] [y (assert_equal_1/y:0) = ] [1]
         [[{{function_node assert_equal_1_Assert_AssertGuard_false_274}}{{node assert_equal_1/Assert/AssertGuard/Assert}}]]
        at org.tensorflow.internal.c_api.AbstractTF_Status.throwExceptionIfNotOK(AbstractTF_Status.java:87)
        at org.tensorflow.Session.run(Session.java:850)
        at org.tensorflow.Session.access$300(Session.java:82)
        at org.tensorflow.Session$Runner.runHelper(Session.java:552)
        at org.tensorflow.Session$Runner.runNoInit(Session.java:499)
        at org.tensorflow.Session$Runner.run(Session.java:495)
        at org.tensorflow.SessionFunction.call(SessionFunction.java:116)
        at org.tensorflow.SavedModelBundle.call(SavedModelBundle.java:481)
        at org.vitrivr.cineast.core.features.CLIPText.embedText(CLIPText.java:111)
        at org.vitrivr.cineast.core.features.CLIPText.getSimilar(CLIPText.java:94)
        at org.vitrivr.cineast.standalone.runtime.RetrievalTask.call(RetrievalTask.java:57)
        at org.vitrivr.cineast.standalone.runtime.RetrievalTask.call(RetrievalTask.java:14)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        ... 1 more