search

akhawaja / ksuid

Java implementation of K-Sortable Globally Unique IDs
Other
41 stars 9 forks source link

Generated ksuids are not portable #2

Open kosprov opened 5 years ago

kosprov commented 5 years ago

Hi,

I just used the original Go implementation to create a KSUID and I got:

1GZvgyPvVmuiQMTuzkEtKSogs2V

The Java implementation gave me:

COH2CXgKpgJ9Bvn8SF1xUa9CeTt

The timestamp part is clearly not similar even though they were generated seconds apart.

When I try to parse 1GZvgyPvVmuiQMTuzkEtKSogs2V with the Java implementation I get:

Time: 1991-03-09T10:48:35Z[UTC]
Timestamp: 668515715
Payload: [35, -17, 86, 107, -94, 64, -60, -18, -50, 65, 45, 41, 42, 32, -77, 101]

Similarly, when I parse COH2CXgKpgJ9Bvn8SF1xUa9CeTt with the Go implementation I get:

REPRESENTATION:

  String: COH2CXgKpgJ9Bvn8SF1xUa9CeTt
     Raw: 56D88ECF863A0F0D8192383AB8243D790C90248D

COMPONENTS:

       Time: 2060-07-14 14:15:11 +0200 EET
  Timestamp: 1457032911
    Payload: 863A0F0D8192383AB8243D790C90248D

This library must be interoperable with other implementations.

akhawaja commented 5 years ago

Hi @kosprov. Thank you for bringing this to my attention. I'll take a deeper look at this. Cheers!

akhawaja commented 5 years ago

Hi @kosprov. What is the epoch time set to in your Go implementation? Something to keep in mind is that there is a subtract operation performed on the generated timestamp prior to Base62 conversion. Another point to note is that the Base62 conversion in the Go implementation have the ASCII characters in a different order. Just some thoughts. Thank you.

kosprov commented 5 years ago

I used the original Go implementation from Segment and a JavaScript implementation. Those two are interoperable. Both use the epoch mentioned in the blog post (14e8 seconds / 14e11 millis)

akhawaja commented 5 years ago

Thank you for the solution references. While interoperability with other implementations was not an original goal, I can see its value. As time permits, I'll work towards making this library compatible with other implementations starting with the ones you have referenced. Cheers!

akhawaja commented 5 years ago

Hi @kosprov. I've published a version of this library to Maven Central that should generate identifiers that are portable. Since this is a breaking change, I am incrementing the version number to a 2.x.x release.

I'd like your feedback. When possible, can you please test the new version and compare your results to the Go and Node.js implementations then report back if the changes are acceptable?

In Maven Central, you can find the build as version 2.0.0-RC1. In GitHub, you can find the code under the branch feature/portable-ksuid.

Thank you.

yanok commented 3 years ago

I'm not the one who asked for it originally but I can confirm that with 2.0.0-RC1 version I get UIDs that look close enough to the ones from Go implementation.