High CPU usage after including External Task Client Spring Boot Starter

[MiguelV-dev]

Environment (Required on creation)

Camunda external task client, all supported versions.

OS: both Ubuntu 20.04.6 LTS and MacOS Ventura

Java: openjdk version "17.0.6" 2023-01-17 OpenJDK Runtime Environment (build 17.0.6+10-Ubuntu-0ubuntu120.04.1) OpenJDK 64-Bit Server VM (build 17.0.6+10-Ubuntu-0ubuntu120.04.1, mixed mode, sharing)

Spring Boot: 2.6.7

Description (Required on creation; please attach any relevant screenshots, stacktraces, log files, etc. to the ticket)

Using the External Task Client Spring Boot Starter (camunda-bpm-spring-boot-starter-external-task-client) causes High CPU usage when there are no subscriptions defined.

Steps to reproduce (Required on creation)

Just follow the instructions on https://docs.camunda.org/manual/7.18/user-guide/ext-client/spring-boot-starter/#handler-configuration-example but don't define handlers yet.

Observed Behavior (Required on creation)

• High CPU usage (100%)

Expected behavior (Required on creation)

• "Normal" CPU usage.

Root Cause (Required on prioritization)

If the dependency is added with the required client configurations but there are no subscription handlers (ExternalTaskHandler) defined, the TopicSubscriptionManager's runBackoffStrategy is not executed and the acquire method will run without any break, even though there are no subscriptions at all.

Solution Ideas

• run the runBackoffStrategy even without subscriptions/handlers
• don't auto start the TopicSubscriptionManager's thread if there are no subscriptions/handlers.

Hints

The method being called was discovered after performing some profiling:

Using the debugger, the list of subscriptions is always empty:

Links

https://docs.camunda.org/manual/7.18/user-guide/ext-client/spring-boot-starter/#handler-configuration-example https://forum.camunda.io/t/external-task-high-cpu-usage/43857

[danielkelemen]

Hi @skamv, thank you for opening the issue! Which version of the external task client are you using? Did you experience this issue after some environment change? For instance, after updating Java.

[MiguelV-dev]

I'm using the version 7.18.0 I was using the camunda-external-task-client so I was refactoring the code to use the camunda-bpm-spring-boot-starter-external-task-client. I followed the user guide for it, so it's basically a fresh implementation, it's not a result of upgrading some other external artifact.

[MiguelV-dev]

Hi @danielkelemen, did you manage to reproduce this issue? thank you

[FlorianCassayre]

@danielkelemen any updates on that? In my opinion this is a major issue that prevents any serious use of the external task client for Spring Boot.

[danielkelemen]

Hi @skamv & @FlorianCassayre, I tried to reproduce the problem with JDK 17 and the 7.18 loan-granting-spring-boot-webapp example on MacOS but I didn't experience any unusual CPU usage. This example uses spring boot 2.7.3 and defines multiple ExternalTaskSubscription so it should probably reproduce the problem. Does the problem happen to you if you run this example?

[FlorianCassayre]

Hi @danielkelemen, thanks for looking into it. The example that you provided works correctly as is. However, if you comment out the three @ExternalTaskSubscription beans and re-run the application, you will be able to observe the problem that @skamv has reported. Basically if there are no external tasks to be subscribed to, the client will hog the CPU resources on one thread.

[danielkelemen]

@FlorianCassayre, thank you, that reproduces the problem indeed.

In this case it's because the runBackoffStrategy is skipped due to the subscriptions (and thus taskTopicRequests) being empty and the acquire just runs without limits. I don't think the PR with the closable client fixes this though. I think the fix is to run the backoff strategy, even if there are no subscriptions.

[FlorianCassayre]

Indeed, that is a different problem -- that could be fixed in the same PR nevertheless.

[danielkelemen]

For me it's still not clear how this happens with this example since that does have some subscriptions. I guess, it could happen if the example is not completed entirely.

Anyway, I will update the ticket to make it clearer that this happens when subscriptions are empty.

[danielkelemen]

@ThorbenLindhauer, assigning for decision/prioritization.

[ThorbenLindhauer]

Thanks. We will not schedule this ticket for the time being, because we have other priorities. We'd apprecitate a contribution nevertheless.

I will also go ahead with closing this ticket as it is a duplicate of https://github.com/camunda/camunda-bpm-platform/issues/2337