search

mbari-org / cthulhu

Cross-platform video player that supports annotation localizations (i.e. bounding boxes)
https://mbari-media-management.github.io/cthulhu/
Other
4 stars 2 forks source link

Error when adding localizations #17

Closed kevinsbarnard closed 2 years ago

kevinsbarnard commented 2 years ago

Hi Brian, I'm running into an issue when adding localizations via ZMQ (sharktopoda client localization controller). For reference, I'm running Cthulhu 1.0.1 (e5e1eb6) via ./gradlew run with Java 17.

What I'm doing:

  1. Send a "clear" message to the Localization IO
  2. Send a first localization (this appears to run without error, but I'm still not seeing the localization show up in the player)
  3. Send a second localization --> error

The relevant Cthulhu log I'm seeing is as follows:

10:39:20.368 [Thread-5                 ] DEBUG o.m.v.sharktopoda.client.localization.IO | Received on 'localization': {"action": "clear", "localizations": []}
10:39:25.377 [Thread-5                 ] DEBUG o.m.v.sharktopoda.client.localization.IO | Received on 'localization': {"action": "add", "localizations": [{"concept": "Crossaster", "elapsed_time_millis": 133, "durationMillis": 0, "videoReferenceUuid": "52dc758b-43ee-4a1c-b650-3322febb8d5a", "annotationUuid": "424c8cee-7b65-4cb3-bf80-213480d836b6", "localizationUuid": "05bef4dc-100e-4313-a9a0-14fe8585b78e", "x": 1056, "y": 952, "width": 72, "height": 68, "sourceId": null}]}
10:39:25.378 [Thread-5                 ] DEBUG o.m.v.s.c.l.LocalizationController       | Adding localization (uuid = 05bef4dc-100e-4313-a9a0-14fe8585b78e)
10:39:30.383 [Thread-5                 ] DEBUG o.m.v.sharktopoda.client.localization.IO | Received on 'localization': {"action": "add", "localizations": [{"concept": "Crossaster", "elapsed_time_millis": 166, "durationMillis": 0, "videoReferenceUuid": "52dc758b-43ee-4a1c-b650-3322febb8d5a", "annotationUuid": "182b6b2e-d415-43fe-9698-e3c11c18f990", "localizationUuid": "d3979f81-f57c-47d1-b81d-4d5b7b9205d7", "x": 1062, "y": 956, "width": 70, "height": 66, "sourceId": null}]}
10:39:30.384 [Thread-5                 ] DEBUG o.m.v.s.c.l.LocalizationController       | Adding localization (uuid = d3979f81-f57c-47d1-b81d-4d5b7b9205d7)
Exception in thread "Thread-5" java.lang.NullPointerException: Cannot invoke "java.lang.Comparable.compareTo(Object)" because the return value of "java.util.function.Function.apply(Object)" is null
        at java.base/java.util.Comparator.lambda$comparing$77a9974f$1(Comparator.java:473)
        at javafx.base/javafx.collections.transformation.SortedList$ElementComparator.compare(SortedList.java:299)
        at javafx.base/javafx.collections.transformation.SortedList$ElementComparator.compare(SortedList.java:288)
        at java.base/java.util.Arrays.binarySearch0(Arrays.java:2340)
        at java.base/java.util.Arrays.binarySearch(Arrays.java:2325)
        at javafx.base/javafx.collections.transformation.SortedList.findPosition(SortedList.java:331)
        at javafx.base/javafx.collections.transformation.SortedList.insertToMapping(SortedList.java:336)
        at javafx.base/javafx.collections.transformation.SortedList.addRemove(SortedList.java:406)
        at javafx.base/javafx.collections.transformation.SortedList.sourceChanged(SortedList.java:108)
        at javafx.base/javafx.collections.transformation.TransformationList.lambda$getListener$0(TransformationList.java:106)
        at javafx.base/javafx.collections.WeakListChangeListener.onChanged(WeakListChangeListener.java:88)
        at javafx.base/com.sun.javafx.collections.ListListenerHelper$SingleChange.fireValueChangedEvent(ListListenerHelper.java:164)
        at javafx.base/com.sun.javafx.collections.ListListenerHelper.fireValueChangedEvent(ListListenerHelper.java:73)
        at javafx.base/javafx.collections.ObservableListBase.fireChange(ObservableListBase.java:239)
        at javafx.base/javafx.collections.FXCollections$UnmodifiableObservableListImpl.lambda$new$0(FXCollections.java:963)
        at javafx.base/javafx.collections.WeakListChangeListener.onChanged(WeakListChangeListener.java:88)
        at javafx.base/com.sun.javafx.collections.ListListenerHelper$SingleChange.fireValueChangedEvent(ListListenerHelper.java:164)
        at javafx.base/com.sun.javafx.collections.ListListenerHelper.fireValueChangedEvent(ListListenerHelper.java:73)
        at javafx.base/javafx.collections.ObservableListBase.fireChange(ObservableListBase.java:239)
        at javafx.base/javafx.collections.ListChangeBuilder.commit(ListChangeBuilder.java:482)
        at javafx.base/javafx.collections.ListChangeBuilder.endChange(ListChangeBuilder.java:541)
        at javafx.base/javafx.collections.ObservableListBase.endChange(ObservableListBase.java:211)
        at javafx.base/javafx.collections.ModifiableObservableListBase.add(ModifiableObservableListBase.java:162)
        at java.base/java.util.AbstractList.add(AbstractList.java:111)
        at vcr4j.sharktopoda.client@4.4.1.jre11-SNAPSHOT/org.mbari.vcr4j.sharktopoda.client.localization.LocalizationController.addOrReplaceLocalizationInternal(LocalizationController.java:108)
        at vcr4j.sharktopoda.client@4.4.1.jre11-SNAPSHOT/org.mbari.vcr4j.sharktopoda.client.localization.LocalizationController.addOrReplaceLocalizationsInternal(LocalizationController.java:81)
        at io.reactivex.rxjava2@2.2.21/io.reactivex.internal.observers.LambdaObserver.onNext(LambdaObserver.java:63)
        at io.reactivex.rxjava2@2.2.21/io.reactivex.internal.operators.observable.ObservableMap$MapObserver.onNext(ObservableMap.java:62)
        at io.reactivex.rxjava2@2.2.21/io.reactivex.internal.operators.observable.ObservableFilter$FilterObserver.onNext(ObservableFilter.java:52)
        at io.reactivex.rxjava2@2.2.21/io.reactivex.internal.operators.observable.ObservableMap$MapObserver.onNext(ObservableMap.java:62)
        at io.reactivex.rxjava2@2.2.21/io.reactivex.internal.operators.observable.ObservableFilter$FilterObserver.onNext(ObservableFilter.java:52)
        at io.reactivex.rxjava2@2.2.21/io.reactivex.subjects.PublishSubject$PublishDisposable.onNext(PublishSubject.java:308)
        at io.reactivex.rxjava2@2.2.21/io.reactivex.subjects.PublishSubject.onNext(PublishSubject.java:228)
        at io.reactivex.rxjava2@2.2.21/io.reactivex.subjects.SerializedSubject.onNext(SerializedSubject.java:104)
        at vcr4j.sharktopoda.client@4.4.1.jre11-SNAPSHOT/org.mbari.vcr4j.sharktopoda.client.localization.IO.lambda$new$2(IO.java:106)
        at java.base/java.lang.Thread.run(Thread.java:833)

From what I can tell, this stems from org.mbari.vcr4j.sharktopoda.client.localization.LocalizationController L108, where the localization is added to the internal JavaFX ObservableList. However, this only happens when adding the second localization and any subsequent localizations.

Any thoughts?

hohonuuli commented 2 years ago

It looks like you're adding a localization without an elapsedTime field (or elapsedTimeMillis field if using JSON)

kevinsbarnard commented 2 years ago

That did the trick- thanks Brian! I was serializing to elapsed_time_millis.