Go library for generating Time-Sorted Unique Identifiers (TSID) Implementation inspired from f4b6a3/tsid-creator
It brings together ideas from Twitter's Snowflake and ULID Spec.
In summary:
Recommended readings:
Create a TSID Factory:
tsidFactory, err := TsidFactoryBuilder().
WithNodeBits(nodeBits). // max 20
WithNode(nodeId). // max 2^nodeBits
WithCustomEpoch(customEpoch).
WithClock(clock).
WithRandom(random).
Build()
Default node Id is 0 & node bits is 0. Epoch and clock are optional, and Random when not provided it internally uses random from math/rand to generate random values
[!NOTE]
.Build()
creates / returns the existing instance of the tsid factory, this is useful where single instance of tsid factory needs to be shared across go routines. But if you need a new instance use.NewInstance()
Generate TSID
tsid, err := tsidFactory.Generate()
Get TSID as int64
:
number := tsid.ToNumber(); // 122390644586507544
Get TSID as string
:
tsidStr := tsid.ToString(); // 03CPHMJ76HV8R
The TSID generator is thread-safe.
Run the following command:
go get github.com/vishal-bihani/go-tsid
The tsid.ToNumber()
method simply unwraps the internal int64
value of a TSID.
id := tsid.ToNumber()
Sequence of TSIDs:
38352658567418867
38352658567418868
38352658567418869
38352658567418870
38352658567418871
38352658567418872
38352658567418873
38352658567418874
38352658573940759 < millisecond changed
38352658573940760
38352658573940761
38352658573940762
38352658573940763
38352658573940764
38352658573940765
38352658573940766
^ ^ look
|--------|------|
time random
The tsid.ToString()
method encodes a TSID to Crockford's base 32 encoding. The returned string is 13 characters long.
idStr := tsid.ToString();
Sequence of TSID strings:
01226N0640J7K
01226N0640J7M
01226N0640J7N
01226N0640J7P
01226N0640J7Q
01226N0640J7R
01226N0640J7S
01226N0640J7T
01226N0693HDA < millisecond changed
01226N0693HDB
01226N0693HDC
01226N0693HDD
01226N0693HDE
01226N0693HDF
01226N0693HDG
01226N0693HDH
^ ^ look
|-------|---|
time random
The string format can be useful for languages that store numbers in double-precision 64-bit binary format IEEE 754, such as Javascript.
The term TSID stands for (roughly) Time-Sorted ID. A TSID is a number that is formed by the creation time along with random bits.
The TSID has 2 components:
The time component is the count of milliseconds since 2020-01-01 00:00:00 UTC.
The Random component has 2 sub-parts:
The counter bits depend on the node bits. If the node bits are 10, the counter bits are limited to 12. In this example, the maximum node value is 2^10-1 = 1023 and the maximum counter value is 2^12-1 = 4095. So the maximum TSIDs that can be generated per millisecond is 4096.
The node identifier uses 10 bits of the random component by default in the TsidFactory
. It's possible to adjust the node bits to a value between 0 and 20. The counter bits are affected by the node bits.
This is the default TSID structure:
adjustable
<---------->
|------------------------------------------|----------|------------|
time (msecs since 2020-01-01) node counter
42 bits 10 bits 12 bits
- time: 2^42 = ~69 years or ~139 years (with adjustable epoch)
- node: 2^10 = 1,024 (with adjustable bits)
- counter: 2^12 = 4,096 (initially random)
Notes:
The node is adjustable from 0 to 20 bits.
The node bits affect the counter bits.
The time component can be used for ~69 years if stored in a SIGNED 64 bits integer field.
The time component can be used for ~139 years if stored in a UNSIGNED 64 bits integer field.
The time component can be 1 ms or more ahead of the system time when necessary to maintain monotonicity and generation speed.
A simple way to avoid collisions is to make sure that each generator has its exclusive node ID. A "node" as we call it in this library can be a physical machine, a virtual machine, a container, a k8s pod, a running process, a database instance number, etc.
Notes:
hostname
manual:
--ip-address
or -i
(lowercase): Display the network address(es) of the host name. Note that this works only if the host name can be resolved. Avoid using this option; use hostname --all-ip-addresses instead.--all-ip-addresses
or -I
(uppercase): Display all network addresses of the host. This option enumerates all configured addresses on all network interfaces. The loopback interface and IPv6 link-local addresses are omitted. Contrary to option -i, this option does not depend on name resolution. Do not make any assumptions about the order of the output.Create a quick TSID:
tsid := tsid.Fast();
Create a quick TSID from canonical string (13 chars)
tsid := tsid.FromString("03CPHMJ76HV8R")
Get the creation unix millis of the tsid
millis := tsid.GetUnixMillis()
A TsidFactory
with a FIXED node identifier and CUSTOM node bits:
// setup a factory for up to 64 nodes and 65536 ID/ms.
tsidFactory, err := TsidFactoryBuilder()
.WithNodeBits(6) // max: 20
.WithNode(63) // max: 2^nodeBits
.Build();
// use the factory
tsid, err := tsidFactory.Generate()
A TsidFactory
with a CUSTOM epoch:
epoch := time.Now().UnixMilli()
tsidFactory, err := TsidFactoryBuilder().
WithCustomEpoch(epoch).
build();
// use the factory
tsid, err := tsidFactory.Generate()
A TsidFactory
with Crypto Random value generator
// using crypto/rand for security use cases
supplier := NewCryptoRandomSupplier()
// creating random value generator
intRandom := NewIntRandom(supplier)
tsidFactory, err := TsidFactoryBuilder().
WithRandom(intRandom).
build();
You can use custom random value suppliers either by implementing
IntSupplier
,ByteSupplier
or usingNewIntRandomWithSupplierFunc
Ports and forks:
Language | Name |
---|---|
Java | vladmihalcea/hypersistence-tsid |
.NET | kgkoutis/TSID.Creator.NET |
PHP | odan/tsid |
Python. | luismedel/tsid-python |
Other OSS:
Language | Name |
---|---|
Java | fillumina/id-encryptor |
.NET | ullmark/hashids.net |
This library is Open Source software released under the Apache-2.0 license.