Different host instances, run for a period of time, the KEY sequence is inconsistent

[ArvinHsieh]

Hi First of all, thank you for providing such a good package. I have some problems in use and would like to ask how to solve it. details as following.

I use this package on one application and deploy this application to two different hosts, Provide KEY generation service through Load Balance architecture. The system local time of the two hosts is calibrated through NTP and uses different GeneratorId.

Instance create method:

private static IIdGenerator _generator;

generator = new IdGenerator(_machineId, new IdGeneratorOptions(sequenceOverflowStrategy: SequenceOverflowStrategy.SpinWait));

But when my application runs for a period of time, the KEY sequence generated on the two hosts will be wrong, as follows.

1.Host A GenKey=700000000000100000 2.Host B GenKey=700000000000200000 3.Host A GenKey=700000000000300000 4.Host B GenKey=700000000000400000 5.Host A GenKey=700000000000500000 6.Host B GenKey=700000000000600000 after a while....... 1001.Host A GenKey=700000000001100000 1002.Host B GenKey=700000000001200000 1003.Host A GenKey=700000000001400000 1004.Host B GenKey=700000000001300000 1005.Host A GenKey=700000000001600000 1006.Host B GenKey=700000000001500000

I hope that the time generated by the two hosts will not be rolled back. How should I deal with it? Maybe implement ITimeSource the NTPTimeSource?

Thanks

[RobThree]

Are you sure (i.e. did you confirm) the _machineId differs on each machine?

static void Main(string[] args)
{
    var a = new IdGenerator(1, new IdGeneratorOptions(sequenceOverflowStrategy: SequenceOverflowStrategy.SpinWait));
    var b = new IdGenerator(2, new IdGeneratorOptions(sequenceOverflowStrategy: SequenceOverflowStrategy.SpinWait));

    for (int i = 0; i < 10; i++)
    {
        var a_id = a.CreateId();
        var b_id = b.CreateId();

        Console.WriteLine($"A: {a_id} {Convert.ToString(a_id, 2)}");
        Console.WriteLine($"B: {b_id} {Convert.ToString(b_id, 2)}");
    }
}

As you can see in the above screenshot the red part contains the _machineId (10 bits), the blue the sequence number (12 bits) and the green part the timestamp.

The values you show seem hand-crafted (probably to demonstrate the problem). Can you please show actual values?

[RobThree]

Oh, wait, nevermind. I think I understood the question wrong. I think I understand now. But I will still need actual generated values (and the corresponding generator (or "machine") id along with it) to see if there's an actual problem or not.

I think what you're seeing is correct. Only part of the ID is a timestamp. There's no guarantee the ID's are all sequential since the machines/generators don't communicate between eachother. The only 'guarantee' you have, concerning ID sequences, is that for a single generator, the values are sequential. IdGen also guarantees that there are no collisions (as long as no sequence overflows occur and machine/generator ID's are different where they need to be etc.) without actually having to coordinate the IdGenerators or requiring communication between those generators. So ID's generated by different generators (on different machines) may not be, and most likely won't be, 100% sequential.

[ArvinHsieh]

Oh, wait, nevermind. I think I understood the question wrong. I think I understand now. But I will still need actual generated values (and the corresponding generator (or "machine") id along with it) to see if there's an actual problem or not.

I think what you're seeing is correct. Only part of the ID is a timestamp. There's no guarantee the ID's are all sequential since the machines/generators don't communicate between eachother. The only 'guarantee' you have, concerning ID sequences, is that for a single generator, the values are sequential. IdGen also guarantees that there are no collisions (as long as no sequence overflows occur and machine/generator ID's are different where they need to be etc.) without actually having to coordinate the IdGenerators or requiring communication between those generators. So ID's generated by different generators (on different machines) may not be, and most likely won't be, 100% sequential.

Thank you for your explanation, the actual _machineId(GeneratorId) I use is 601/604, but if I want to ensure that the serial numbers generated are continuous in this scenario, is there any suggestion? For example, I need to store the ID generated each time in Redis and check whether they need to be regenerated. But this will have an impact on the generation speed for concurrent accesses.

Thanks for your reply.

[RobThree]

If you need sequential, "continuous", ID's then IdGen is not for you. IdGen is, as the first line in the documentation states: "... a low-latency, distributed, uncoordinated, (roughly) time ordered, compact and highly available Id generation system". For Id's to be continuous you always need some form of coordination.

[ArvinHsieh]

Ok, I understood. Thanks.