quartznet / quartznet

Quartz Enterprise Scheduler .NET
http://www.quartz-scheduler.net/
Apache License 2.0
6.51k stars 1.69k forks source link

Wrong PersistentStore is used when creating multiple Schedulers #2218

Closed Roiskia closed 2 months ago

Roiskia commented 10 months ago

Describe the bug

I am trying to instantiate multiple, independent schedulers with their own persistent stores. But after creating more then one, they all seem to use the same.

I am not sure what I am doing something wrong or if I have missed something in the documentation about this and would appreciate any advice.

Version used

Quartz.Net 3.7.0

To Reproduce

I parse a list of configured datapools, make sure they are unique, and then instantiate a scheduler for each of them like this:

string datapool = "example";

var factory = SchedulerBuilder.Create()
    .WithId(Guid.NewGuid().ToString())
    .WithName(datapool)
    .UsePersistentStore(opt => GetConStr(opt, datapool))
    .Build();

var scheduler = await factory.GetScheduler();

The GetConStr method determines what type of database the datapool is using and calls the appropriate extension method on the PersistantStoreOptions object with the datapool's connection string (such as UseOracle/UseSqlServer).

For example, Datapool A might use SQL Server and Datapool B might use Oracle. Using only Datapool A will write to the SQL Server connection. Using only Datapool B will write to the Oracle connection. Using both A and B in this order will result in both writing to the Oracle connection.

Expected behavior

Using both Datapools A and B in this order will cause the first scheduler to write to SQL Server und the second one to write to Oracle.

lahma commented 10 months ago

Can you create a complete sample, this is just one configuration. You can create check what kind of configuration you have with:

var pool1 = SchedulerBuilder.Create()
    .WithId(Guid.NewGuid().ToString())
    .WithName("pool1");

foreach (string key in pool1.Properties.Keys)
{
    Console.WriteLine($"{key}: {pool1.Properties[key]}");
}

SchedulerBuilder is just a strongly-typed wrapper over generic key-value configuration.

Roiskia commented 10 months ago

Sure, here is a more complete example of what I am doing. As explained earlier, I am creating a scheduler for each unique datapool. The log output seems to be correct at this point.

foreach (var datapool in datapools)
{
    if (_datapools.ContainsKey(datapool))
    {
        continue;
    }

    var builder = SchedulerBuilder.Create()
    .WithId(Guid.NewGuid().ToString())
    .WithName(datapool)
    .UsePersistentStore(opt => GetConStr(opt, datapool))

    StringBuilder stringBuilder = new StringBuilder();
    stringBuilder.AppendLine("Scheduler config:");
    foreach(string key in builder.Properties.Keys)
    {
        stringBuilder.AppendLine($"{key}: {builder.Properties[key]}");
    }
    var config = stringBuilder.ToString();
    _logger.Debug(config);

    var factory = builder.Build();

    var scheduler = await factory.GetScheduler();

    _schedulers.Add(datapool, scheduler);
}
lahma commented 10 months ago

So what's the output? Please hide credentials and other sensitive data from connection strings.

Roiskia commented 10 months ago

The output looks like this:

quartz.scheduler.instanceId: a9631b10-7706-4879-bf1d-e68de08d233a
quartz.scheduler.instanceName: datapool-a
quartz.jobStore.type: Quartz.Impl.AdoJobStore.JobStoreTX, Quartz
quartz.jobStore.useProperties: true
quartz.jobStore.driverDelegateType: Quartz.Impl.AdoJobStore.SqlServerDelegate, Quartz
quartz.jobStore.dataSource: default
quartz.dataSource.default.provider: SqlServer
quartz.dataSource.default.connectionString: data source=********;password=********;user id=********;
quartz.serializer.type: Quartz.Simpl.JsonObjectSerializer, Quartz.Serialization.Json
quartz.jobStore.misfireThreshold: 60000
quartz.scheduler.interruptJobsOnShutdown: True
quartz.scheduler.interruptJobsOnShutdownWithWait: True
quartz.scheduler.batchTriggerAcquisitionFireAheadTimeWindow: 1
quartz.scheduler.instanceId: fe07fdaa-2b71-4816-9cf2-049f60d40d9a
quartz.scheduler.instanceName: datapool-b
quartz.jobStore.type: Quartz.Impl.AdoJobStore.JobStoreTX, Quartz
quartz.jobStore.useProperties: true
quartz.jobStore.driverDelegateType: Quartz.Impl.AdoJobStore.OracleDelegate, Quartz
quartz.jobStore.dataSource: default
quartz.dataSource.default.provider: OracleODPManaged
quartz.dataSource.default.connectionString: data source=********;password=********;user id=********;
quartz.serializer.type: Quartz.Simpl.JsonObjectSerializer, Quartz.Serialization.Json
quartz.jobStore.misfireThreshold: 60000
quartz.scheduler.interruptJobsOnShutdown: True
quartz.scheduler.interruptJobsOnShutdownWithWait: True
quartz.scheduler.batchTriggerAcquisitionFireAheadTimeWindow: 1
lahma commented 10 months ago

Seems that configuration have different values so calling builder.Build() for each should create different scheduler factories producing schedulers with different dbs.

Roiskia commented 10 months ago

I understand that. And I expected that. But I can reliably reproduce the problem described.

If I build schedulers A and B in this order, A will write with the connection from B. I also get data with both SCHED_NAME values in B.

If I build B first and then A, I end up with the same problem, but now both are writing to A.

Even though I get the correct config output for each scheduler.

If I build only A, it will write to A. If I build only B, it will write to B.

Roiskia commented 10 months ago

I have prepared a sample project (see attached zip) that reproduces the problem for you. For simplicity, I am trying to connect to two different Oracle instances in this case.

QuartzExample.zip

Roiskia commented 10 months ago

Today I noticed a github notification for a post by Uwe Laas on this topic, written on 13.12.2023, which is no longer there. Or at least I cannot see it. Is this the root of the problem or does it no longer apply?

Sorry to jump in here, had the same problem while writing tests - isn't it the case that the SchedulerFactory uses the static 'Instance' of SchedulerRepository? I thought it was impossible to create multiple different schedulers within the same process, at least without using dirty tricks like reflection. So, always the same scheduler => always the same store. Or am I wrong?

JezhikLaas commented 10 months ago

Sorry, I deleted my comment because I completely misunderstood the problem.

Am 15.12.23 um 09:30 schrieb Roiskia:

Today I noticed a github notification for a post by Uwe Laas on this topic, written on 13.12.2023, which is no longer there. Or at least I cannot see it. Is this the root of the problem or does it no longer apply?

Sorry to jump in here, had the same problem while writing tests - isn't it the case that the SchedulerFactory uses the static 'Instance' of SchedulerRepository? I thought it was impossible to create multiple different schedulers within the same process, at least without using dirty tricks like reflection. So, always the same scheduler => always the same store. Or am I wrong?

— Reply to this email directly, view it on GitHubhttps://github.com/quartznet/quartznet/issues/2218#issuecomment-1857478313, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AG5DAI2HZGO45J3K6EMRJG3YJQDC3AVCNFSM6AAAAABARRAGKSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJXGQ3TQMZRGM. You are receiving this because you commented.Message ID: @.***>

-- Uwe Laas

Tel 05221 1718677

Laas IT Consulting

https://www.laas-it-consulting.de

Roiskia commented 10 months ago

This is fine. I just wanted to ask because it sounded like a plausible cause of the problem.

Sorry, I deleted my comment because I completely misunderstood the problem.

Roiskia commented 10 months ago

Today i managed to debug the whole process and it seems that the Quartz.Util.DBConnectionManager singleton is the problem here. As the factories within the process will all be using the same internal provider dictionary, they will also be overriding each other's default provider. If I set a dataSourceName on the UseX methods, I can avoid this for now.

The only problem I have here is that this is not obvious at all. Since I am creating new factories, I would not expect them to share any state.

lahma commented 2 months ago

V4 will make data source name required to reduce the chance for these kinds of problems.