quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.54k stars 2.61k forks source link

To be able to schedule clusterized tasks and non-clusterized (in memory) timers #41954

Closed vondacho closed 1 month ago

vondacho commented 1 month ago

Description

Given this fact:

If you add the quarkus-quartz dependency to your project the lightweight scheduler implementation from the quarkus-scheduler extension is automatically disabled.

With a clustered setup of Quartz in a multi-pods environment, it is no more possible to schedule in-memory timers or recurrent in-memory tasks in a given pod in order to trigger actions on some beans, using the @Scheduled annotation.

Does currently exist a recipe to mixt both needs in an elegant way? Or, is it possible to plan some development on the Quarkus-Quartz extension?

Best regards.

Implementation ideas

quarkus-bot[bot] commented 1 month ago

/cc @manovotn (scheduler), @mkouba (scheduler)

vsevel commented 1 month ago

basically the need is to use quartz for some jobs, while keeping the normal local scheduler for all others. it would be nice to have a specific annotation QuartzScheduled to specifically mark some jobs as run by quartz, while all others would still be run through the default local non persistent scheduler. we use quartz for the specific jobs that need to be clustered. and we do not want all other normal @Scheduled jobs to start running through quartz (and even less in a clustered aware situation) just because we added the the quartz extension. If I compare with JobRunr, we can see that they opted for a dedicated annotation. would that be an option for the quartz extension? or is there a way to force the other @Scheduled methods to run local (ie: not run in a clustered aware setup)?

melloware commented 1 month ago

I would think a clustered=false added to the @Schedule annotation would be more elegant no?

mkouba commented 1 month ago

That's an interesting use case. To be clear - this only applies if quarkus.quartz.clustered=true and DB store type.

I don't think that Quartz API has a way to specify that a specific job is clustered or not. It's a global configuration.

So we would need to run the simple in-memory scheduler (quarkus-scheduler) and Quartz scheduler (quarkus-quartz) side by side. Which is not possible at the moment (for example both extensions provide an implementation of io.quarkus.scheduler.Scheduler).

vsevel commented 1 month ago

So we would need to run the simple in-memory scheduler (quarkus-scheduler) and Quartz scheduler (quarkus-quartz) side by side

that would be my preferred choice. I do not want to run everything with quartz.

(for example both extensions provide an implementation of io.quarkus.scheduler.Scheduler

right. that is why I was suggesting providing quartz specific annotations, just like jobrunr does

mkouba commented 1 month ago

So we would need to run the simple in-memory scheduler (quarkus-scheduler) and Quartz scheduler (quarkus-quartz) side by side

that would be my preferred choice. I do not want to run everything with quartz.

I think that we would need a more general solution.

In other words, we would need:

  1. A way to specify on which scheduler impl to execute method/job
  2. A rule to identify the default scheduler impl if (1) is not used
  3. Make sure all scheduler impls can run side by side
vsevel commented 1 month ago

this is very important for us, as we have @Scheduled non cluster aware timers in our base extensions. so any application running quartz, would turn them into cluster aware timers, which is functionally wrong in our case.

mkouba commented 1 month ago

so any application running quartz, would turn them into cluster aware timers, which is functionally wrong in our case.

Yes, any app running Quartz in cluster mode...

mkouba commented 1 month ago

Here's a DRAFT PR https://github.com/quarkusio/quarkus/pull/42129 and usage: https://github.com/quarkusio/quarkus/pull/42129/files#diff-cbfdd765c199e0df9e69a56964fa4cd3e8358aa5e365f4b325295d1b0c2a8525R65-R68.

vsevel commented 1 month ago

fixed by https://github.com/quarkusio/quarkus/pull/42129 thanks a lot @mkouba