Calibration Callbacks not Triggering

[januarydev]

I'm running calibration routines, trying to trigger code based on the calibration succeeding or failing:

drone.getCalibration().getCalibrateGyro()
                        .doOnComplete(() -> Log.d("gyro","complete"))
                        .doOnError(throwable -> Log.d("gyro","error"))
                        .doOnTerminate(() -> Log.d("gyro","terminated")).subscribe();

The drone does start calibrating, but none of the callbacks are triggering upon completion of the calibration routine. Also, on failed calibration, the app as a whole is crashing instead of triggering the onError callback.

Appreciate any ideas. Thanks!

[januarydev]

Here's the calibration failed error:

2020-06-17 09:43:43.308 12660-12737/com.sunlightx.skyfidronecontroller E/AndroidRuntime: FATAL EXCEPTION: grpc-default-executor-2
    Process: com.sunlightx.skyfidronecontroller, PID: 12660
    io.reactivex.exceptions.OnErrorNotImplementedException: FAILED: Failed
        at io.reactivex.internal.functions.Functions$OnErrorMissingConsumer.accept(Functions.java:704)
        at io.reactivex.internal.functions.Functions$OnErrorMissingConsumer.accept(Functions.java:701)
        at io.reactivex.internal.subscribers.LambdaSubscriber.onError(LambdaSubscriber.java:79)
        at io.reactivex.internal.operators.flowable.FlowableDoOnEach$DoOnEachSubscriber.onError(FlowableDoOnEach.java:111)
        at io.reactivex.internal.operators.flowable.FlowableDoOnEach$DoOnEachSubscriber.onError(FlowableDoOnEach.java:111)
        at io.reactivex.internal.util.NotificationLite.accept(NotificationLite.java:221)
        at io.reactivex.internal.operators.flowable.FlowableReplay$BoundedReplayBuffer.replay(FlowableReplay.java:860)
        at io.reactivex.internal.operators.flowable.FlowableReplay$ReplaySubscriber.onError(FlowableReplay.java:406)
        at io.reactivex.internal.operators.flowable.FlowableCreate$BaseEmitter.error(FlowableCreate.java:288)
        at io.reactivex.internal.operators.flowable.FlowableCreate$BufferAsyncEmitter.drain(FlowableCreate.java:531)
        at io.reactivex.internal.operators.flowable.FlowableCreate$BufferAsyncEmitter.tryOnError(FlowableCreate.java:481)
        at io.reactivex.internal.operators.flowable.FlowableCreate$BaseEmitter.onError(FlowableCreate.java:270)
        at io.mavsdk.calibration.Calibration$2.onNext(Calibration.java:310)
        at io.mavsdk.calibration.Calibration$2.onNext(Calibration.java:298)
        at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onMessage(ClientCalls.java:421)
        at io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33)
        at io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:519)
        at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
        at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:764)
     Caused by: io.mavsdk.calibration.Calibration$CalibrationException: FAILED: Failed
        at io.mavsdk.calibration.Calibration$2.onNext(Calibration.java:310) 
        at io.mavsdk.calibration.Calibration$2.onNext(Calibration.java:298) 
        at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onMessage(ClientCalls.java:421) 
        at io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33) 
        at io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33) 
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:519) 
        at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) 
        at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) 
        at java.lang.Thread.run(Thread.java:764) 

Just updated to MAVSDK-Java 0.4.0 too, was working on 0.3.1 up till now but behavior is still the same on the new version.

[JonasVautherin]

Did you update mavsdk-server, too (so that you get mavsdk_server v0.28.0)?

Should I be able to reproduce that with a Pixhawk with your code snippet above?

[januarydev]

Thanks for the quick response!

I updated the server to 0.4.0 as well, I assume that is tied to v28?

I'm actually using your PX4 headless Docker script for testing the app, unsure if the behavior is mirrored on a physical autopilot. I'll see if I can replicate the behavior on a CUAV V5+ this afternoon.

[JonasVautherin]

I updated the server to 0.4.0 as well, I assume that is tied to v28?

Yes, the mavsdk-server package is there to embed mavsdk_server :+1:

I'm actually using your PX4 headless Docker script for testing the app

I think I usually test the calibration with HITL. I'll try to test with SITL!

I'll see if I can replicate the behavior on a CUAV V5+ this afternoon.

Ok good, let me know!

[januarydev]

Just connected to the CUAV V5+ and started a gyro cal with the same code above, callbacks still don't get triggered, although the calibration completes successfully. Doing an accel cal and not moving the autopilot to force a calibration failure also causes an app crash with the real autopilot as well.

[januarydev]

Here's the log from the physical autopilot calibration failure.

2020-06-19 08:33:19.342 22735-22810/com.sunlightx.skyfidronecontroller E/AndroidRuntime: FATAL EXCEPTION: grpc-default-executor-3
    Process: com.sunlightx.skyfidronecontroller, PID: 22735
    io.reactivex.exceptions.OnErrorNotImplementedException: FAILED: Failed
        at io.reactivex.internal.functions.Functions$OnErrorMissingConsumer.accept(Functions.java:704)
        at io.reactivex.internal.functions.Functions$OnErrorMissingConsumer.accept(Functions.java:701)
        at io.reactivex.internal.subscribers.LambdaSubscriber.onError(LambdaSubscriber.java:79)
        at io.reactivex.internal.operators.flowable.FlowableDoOnEach$DoOnEachSubscriber.onError(FlowableDoOnEach.java:111)
        at io.reactivex.internal.operators.flowable.FlowableDoOnEach$DoOnEachSubscriber.onError(FlowableDoOnEach.java:111)
        at io.reactivex.internal.util.NotificationLite.accept(NotificationLite.java:221)
        at io.reactivex.internal.operators.flowable.FlowableReplay$BoundedReplayBuffer.replay(FlowableReplay.java:860)
        at io.reactivex.internal.operators.flowable.FlowableReplay$ReplaySubscriber.onError(FlowableReplay.java:406)
        at io.reactivex.internal.operators.flowable.FlowableCreate$BaseEmitter.error(FlowableCreate.java:288)
        at io.reactivex.internal.operators.flowable.FlowableCreate$BufferAsyncEmitter.drain(FlowableCreate.java:531)
        at io.reactivex.internal.operators.flowable.FlowableCreate$BufferAsyncEmitter.tryOnError(FlowableCreate.java:481)
        at io.reactivex.internal.operators.flowable.FlowableCreate$BaseEmitter.onError(FlowableCreate.java:270)
        at io.mavsdk.calibration.Calibration$2.onNext(Calibration.java:310)
        at io.mavsdk.calibration.Calibration$2.onNext(Calibration.java:298)
        at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onMessage(ClientCalls.java:421)
        at io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33)
        at io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:519)
        at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
        at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:764)
     Caused by: io.mavsdk.calibration.Calibration$CalibrationException: FAILED: Failed
        at io.mavsdk.calibration.Calibration$2.onNext(Calibration.java:310) 
        at io.mavsdk.calibration.Calibration$2.onNext(Calibration.java:298) 
        at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onMessage(ClientCalls.java:421) 
        at io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33) 
        at io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33) 
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:519) 
        at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) 
        at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) 
        at java.lang.Thread.run(Thread.java:764) 

Looks like it is looking for a try/catch exception handler instead of using the onError callback?

[JonasVautherin]

I just made a test with MAVSDK-Python's calibration example, and it works against SITL. Let me try to reproduce with MAVSDK-Java.

[JonasVautherin]

I tried to reproduce in #38. Running against SITL, I get progress. The issue is that it never completes, and therefore just hangs at the end. This sounds like a bug with finite streams in the auto-generation. I'll have a look.

In the meantime, could you try my example #38 and let me know if you get the same error?

I updated the server to 0.4.0 as well, I assume that is tied to v28?

Yes, that should be fine. I don't understand why you get this "not implemented" exception. I'm using java, though, and mavsdk_server from recent develop.

[januarydev]

Alright, ran the new calibration example code with mavsdk-server v28 alongside SITL. Looks like the dependency was still set to mavsdk 0.3.1, so updated that to 0.4.0 as well just to be sure. The calibration completes, but still never seems to signal complete or throw an error, the task just hangs until cancelled. Here's the log:

9:33:24 AM: Executing task 'calibrate'...

> Configure project :

=======
WARNING: this is not the best way to run `jshell`, as it may not be cross-platform and it won't have auto-completion enabled.
You may want to run the following instead:

    $ ./gradlew exportDeps
    $ jshell --class-path $(ls -d build/jshell_classpath_export/* | tr '\n' ':')
=======

> Task :compileJava UP-TO-DATE
> Task :processResources NO-SOURCE
> Task :classes UP-TO-DATE

> Task :calibrate
09:33:24.897 [main] DEBUG io.mavsdk.example.Calibrate - Starting example: calibration...
09:33:25.323 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.0
09:33:26.336 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.05
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/C:/Users/sunli/.gradle/caches/modules-2/files-2.1/com.google.protobuf/protobuf-lite/3.0.1/59b5b9c6e1a3054696d23492f888c1f8b583f5fc/protobuf-lite-3.0.1.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
09:33:27.343 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.1
09:33:28.355 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.15
09:33:29.363 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.2
09:33:30.371 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.25
09:33:31.380 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.3
09:33:32.388 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.35
09:33:33.398 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.4
09:33:34.404 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.45
09:33:35.412 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.5
09:33:36.422 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.55
09:33:37.431 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.6
09:33:38.442 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.65
09:33:39.448 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.7
09:33:40.461 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.75
09:33:41.465 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.8
09:33:42.474 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.85
09:33:43.484 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.9
09:33:44.488 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 0.95
09:33:45.500 [grpc-default-executor-1] DEBUG io.mavsdk.example.Calibrate - Progress: 1.0

> Task :calibrate FAILED

Execution failed for task ':calibrate'.
> Build cancelled while executing task ':calibrate'