Why is thread count setting ignored?

[connollyst]

Hi there,

In the documentation you mention "thread count setting is ignored by the MongoDB store but Quartz requires it". Can you explain why the thread count is not used? Are there any performance or deadlock concerns here?

I feel like database persistence of jobs and the number of threads used to execute jobs simultaneously are separate concerns. Is quartz-mongodb inherently incompatible with a thread pool, or is that just a missing feature?

Thanks, Sean

[michaelklishin]

The answer is: nobody has reported this as a missing feature. I don't think you want concurrent writes performed by a job store but if it's a matter of passing a setting down to Quartz then it's certainly worth supporting. Feel free to look into a pull request.

[connollyst]

Hey Michael,

Including @pwojnowski as he may have some input from the clustering perspective.

The way I see it, you have you org.quartz.jobStore namespace which deals with where and how jobs are persisted. That's what the quartz-mongodb project implements.

Then you've got your org.quartz.threadPool namespace which affects how many Quartz jobs can be executed at once. This is handled by Quartz.

They seem like two separate concerns and I can see how you would want to persist your jobs, but also want to have multiple threads for executing jobs.

You're right, you don't want to have concurrent database reads and writes here, but could synchronize the lowest level database access to prevent threads stepping on each other. As long as your jobs are longer than a few ms and so spend more time executing than accessing the database, you'll gain performance with more threads in the pool.

Here's some documentation showing the configuration of an Oracle jobStore in a clustered environment with 25 threads in the pool.

We can have a look at other job stores and see how they coordinate database access. I think we just need to synchronize reads & writes and then we can increase the threadPool count and Quartz will do it's thing.

I might be thinking about this all wrong, don't hesitate to correct me.

If that sounds correct, I'm happy to contribute a PR. Can I ask, did you base the code on any other jobStores such that the code is structured similarly? That would help understand where they synchronize and where we need to.

Cheers, -Sean

[pwojnowski]

I looked at other JobStores and they do nothing with threadPoolSize parameter, so this parameter seems to be only interpreted by Quartz itself.

I'm not sure which part can be executed concurrently by Quartz. In quartz-mongodb triggers are locked by scheduler instance inside TriggerRunner.acquireNextTriggers() and those successfully locked by the current thread are returned from the method. So, if the same thread is executing its job, it should be fine.

I'm not sure about pausing/resuming jobs, because there is no locking and don't know how Quartz process it.

So, in general, setting threadCount to more than 1 should work for running, but requires testing.

[michaelklishin]

It's not really clear what we should do => closing unless specific suggestions emerge.