Closed budarin closed 2 years ago
Could you explain how UUID is more productive in terms of a DB? ULID is shorter string wise.. I guess you probably mean regarding sharding?
I’m not sure that converting ULID to UUID is within the scope of this library. If you want a UUID, why not use a UUID library?
UUID is stored in Postgres as 16 bytes not as a string with 26 bytes length
@budarin Would you consider making a PR? I'd happily consider the feature if someone were to handle it. It's not on my roadmap at least.
Packages to convert base32 encoded strings to buffers and back already exist:
import { ulid } from 'ulidx';
import { CrockfordBase32 } from 'crockford-base32';
const ulidString = ulid();
console.log('ulid', ulidString);
const buffer = CrockfordBase32.decode(ulidString);
console.log('decoded', new Uint8Array(buffer).toString());
const res = CrockfordBase32.encode(buffer);
console.log('encoded', res);
Result:
ulid 01GA1GBQMCQC2HRD3E2HA9PJRC
decoded 1,130,131,5,222,140,187,5,28,52,110,20,84,155,75,12
encoded 01GA1GBQMCQC2HRD3E2HA9PJRC
I'm adding more info here because I ran into this as well. If you're using Postgres there are benefits to storing ULIDs as UUIDs; Using 16bytes vs 26bytes is one of them. To expand on @samal-rasmussen 's response you can get a UUID string like so
import { ulid } from 'ulidx';
import { CrockfordBase32 } from 'crockford-base32';
import * as uuidBuffer from 'uuid-buffer'
const ulidString = ulid();
console.log("ulid", ulidString)
const buffer = CrockfordBase32.decode(ulidString);
console.log("uuid", uuidBuffer.toString(buffer))
The uuidBuffer.toString function looks pretty simple
function toString(buffer) {
if (buffer.length != 16)
throw new Error("Invalid buffer length for uuid: " + buffer.length);
if (buffer.equals(Buffer.alloc(16)))
return null; // If buffer is all zeros, return null
var str = buffer.toString('hex');
return str.slice(0, 8) + "-" + str.slice(8, 12) + "-" + str.slice(12, 16) + "-" + str.slice(16, 20) + "-" + str.slice(20);
}
source: https://github.com/davidtsai/uuid-buffer
Alternatively you can use Ulid
from id128
found here https://github.com/aarondcohen/id128#ulid
And with that library you can get the uuid representation like so
const { Ulid } = require('id128');
// Returns Buffer
Ulid.generate()
// Returns UUID string
Ulid.generate().toRaw()
// Returns ULID string
Ulid.generate().toCanonical()
That library also has Ulid Monotonic
Packages to convert base32 encoded strings to buffers and back already exist:
import { ulid } from 'ulidx'; import { CrockfordBase32 } from 'crockford-base32'; const ulidString = ulid(); console.log('ulid', ulidString); const buffer = CrockfordBase32.decode(ulidString); console.log('decoded', new Uint8Array(buffer).toString()); const res = CrockfordBase32.encode(buffer); console.log('encoded', res);
Result:
ulid 01GA1GBQMCQC2HRD3E2HA9PJRC decoded 1,130,131,5,222,140,187,5,28,52,110,20,84,155,75,12 encoded 01GA1GBQMCQC2HRD3E2HA9PJRC
@samal-rasmussen It's worth to note that using a monotonic ulid:
const ulidString = ulid.monotonic();
console.log("ulid", ulidString)
const buffer = CrockfordBase32.decode(ulidString);
console.log('decoded', new Uint8Array(buffer).toString());
this will result in
ulid 01H8KGF9HXVJVQSPNGAXE9KG82
decoded 0,98,137,193,233,143,119,45,223,54,172,21,215,38,112,64,128
encoded 01H8KGF9HXVJVQSPNGAXE9KG8200
I'm adding more info here because I ran into this as well. If you're using Postgres there are benefits to storing ULIDs as UUIDs; Using 16bytes vs 26bytes is one of them. To expand on @samal-rasmussen 's response you can get a UUID string like so
import { ulid } from 'ulidx'; import { CrockfordBase32 } from 'crockford-base32'; import * as uuidBuffer from 'uuid-buffer' const ulidString = ulid(); console.log("ulid", ulidString) const buffer = CrockfordBase32.decode(ulidString); console.log("uuid", uuidBuffer.toString(buffer))
The uuidBuffer.toString function looks pretty simple
function toString(buffer) { if (buffer.length != 16) throw new Error("Invalid buffer length for uuid: " + buffer.length); if (buffer.equals(Buffer.alloc(16))) return null; // If buffer is all zeros, return null var str = buffer.toString('hex'); return str.slice(0, 8) + "-" + str.slice(8, 12) + "-" + str.slice(12, 16) + "-" + str.slice(16, 20) + "-" + str.slice(20); }
source: https://github.com/davidtsai/uuid-buffer
Alternatively you can use
Ulid
fromid128
found here https://github.com/aarondcohen/id128#ulidAnd with that library you can get the uuid representation like so
const { Ulid } = require('id128'); // Returns Buffer Ulid.generate() // Returns UUID string Ulid.generate().toRaw() // Returns ULID string Ulid.generate().toCanonical()
That library also has Ulid Monotonic
@stoickeyboard this will fail when using monotonic ulid. In fact while this will work
function uuidBuffer(buffer) {
if (buffer.length != 16)
throw new Error("Invalid buffer length for uuid: " + buffer.length);
if (buffer.equals(Buffer.alloc(16)))
return null; // If buffer is all zeros, return null
var str = buffer.toString('hex');
return str.slice(0, 8) + "-" + str.slice(8, 12) + "-" + str.slice(12, 16) + "-" + str.slice(16, 20) + "-" + str.slice(20);
}
const ulidString = ulid.ulid();
console.log("ulid", ulidString)
const buffer = CrockfordBase32.decode(ulidString);
console.log('decoded', new Uint8Array(buffer).toString());
const res = CrockfordBase32.encode(buffer);
console.log('encoded', res);
console.log("uuid", uuidBuffer(buffer))
getting the right result
ulid 01H8KGKBKZJBXA45SQWS9CGXW8
decoded 0,98,137,194,107,159,228,190,168,133,205,249,148,178,29,226
encoded 01H8KGKBKZJBXA45SQWS9CGXW8
uuid 006289c2-6b9f-e4be-a885-cdf994b21de2
switching to monotonic:
const ulidString = ulid.monotonic();
will cause
ulid 01H8KGV9S2YRCX8J6P2TRRT0JY
decoded 0,98,137,195,105,200,189,134,117,18,53,133,172,99,64,151,128
encoded 01H8KGV9S2YRCX8J6P2TRRT0JY00
(node:83614) UnhandledPromiseRejectionWarning: Error: Invalid buffer length for uuid: 17
Apparently it will fail (randomly) with a UnhandledPromiseRejectionWarning: Error: Invalid buffer length for uuid: 17
even when non monitonic ulid has been generated:
ulid 01H8KGZ0TABWG23ZATB80NFCMS
decoded 0,98,137,195,224,210,151,200,8,127,86,150,128,85,236,166,64
encoded 01H8KGZ0TABWG23ZATB80NFCMS00
(node:84065) UnhandledPromiseRejectionWarning: Error: Invalid buffer length for uuid: 17
using
function uuidBuffer(buffer) {
if (buffer.length != 16)
throw new Error("Invalid buffer length for uuid: " + buffer.length);
if (buffer.equals(Buffer.alloc(16)))
return null; // If buffer is all zeros, return null
var str = buffer.toString('hex');
return str.slice(0, 8) + "-" + str.slice(8, 12) + "-" + str.slice(12, 16) + "-" + str.slice(16, 20) + "-" + str.slice(20);
}
const ulidString = ulid.ulid(); // randomly fails with Error: Invalid buffer length for uuid: 17
console.log("ulid", ulidString)
const buffer = CrockfordBase32.decode(ulidString);
console.log('decoded', new Uint8Array(buffer).toString());
const res = CrockfordBase32.encode(buffer);
console.log('encoded', res);
console.log("uuid", uuidBuffer(buffer))
@loretoparisi in your example you use ulid.monotonic()
. Where is this from? It doesn't seem to be in ulidx.
When I use monotonicFactory from ulidx I get a successfull roundtrip:
import { monotonicFactory } from 'ulidx';
import { CrockfordBase32 } from 'crockford-base32';
const ulid = monotonicFactory();
const ulidString = ulid();
console.log('ulid', ulidString);
const buffer = CrockfordBase32.decode(ulidString);
console.log('decoded', new Uint8Array(buffer).toString());
const res = CrockfordBase32.encode(buffer);
console.log('encoded', res);
ulid 01H8KMSMD9PMXQFA71NMMZCF3X
decoded 1,138,39,76,209,169,181,59,119,168,225,173,41,246,60,125
encoded 01H8KMSMD9PMXQFA71NMMZCF3X
@loretoparisi in your example you use
ulid.monotonic()
. Where is this from? It doesn't seems to be in ulidx.When I use monotonicFactory from ulidx I get a successfull roundtrip:
import { monotonicFactory } from 'ulidx'; import { CrockfordBase32 } from 'crockford-base32'; const ulid = monotonicFactory(); const ulidString = ulid(); console.log('ulid', ulidString); const buffer = CrockfordBase32.decode(ulidString); console.log('decoded', new Uint8Array(buffer).toString()); const res = CrockfordBase32.encode(buffer); console.log('encoded', res);
ulid 01H8KMSMD9PMXQFA71NMMZCF3X decoded 1,138,39,76,209,169,181,59,119,168,225,173,41,246,60,125 encoded 01H8KMSMD9PMXQFA71NMMZCF3X
From the original ulid package, it should be the same. Btw I have actually found that, randomly it fails if you retries more often, like in this case:
ulid 01H8KGZ0TABWG23ZATB80NFCMS
decoded 0,98,137,195,224,210,151,200,8,127,86,150,128,85,236,166,64
encoded 01H8KGZ0TABWG23ZATB80NFCMS00
(node:84065) UnhandledPromiseRejectionWarning: Error: Invalid buffer length for uuid: 17
You can see how apparently const buffer = CrockfordBase32.decode(ulidString);
is adding 0
padding to the decoded string...
The ulid
package? That hasn't been updated in 6 years... ulidx exists because that one is abandoned...
But I tried it and I can't get ulid.monotonic from it either: https://stackblitz.com/edit/js-n52gtt?file=index.js
import * as ulid from 'ulid';
import { CrockfordBase32 } from 'crockford-base32';
function uuidBuffer(buffer) {
if (buffer.length != 16)
throw new Error("Invalid buffer length for uuid: " + buffer.length);
if (buffer.equals(Buffer.alloc(16)))
return null; // If buffer is all zeros, return null
var str = buffer.toString('hex');
return str.slice(0, 8) + "-" + str.slice(8, 12) + "-" + str.slice(12, 16) + "-" + str.slice(16, 20) + "-" + str.slice(20);
}
const ulidString = ulid.monotonic();
console.log("ulid", ulidString)
const buffer = CrockfordBase32.decode(ulidString);
console.log('decoded', new Uint8Array(buffer).toString());
const res = CrockfordBase32.encode(buffer);
console.log('encoded', res);
console.log("uuid", uuidBuffer(buffer))
This just fails with ulid.monotonic is not a function
. It does have ulid.monotonicFactory
and if I use that, then it works just fine. I tried many times.
Maybe just use ulidx?
The
ulid
package? That hasn't been updated in 6 years... ulidx exists because that one is abandoned...But I tried it and I can't get ulid.monotonic from it either: https://stackblitz.com/edit/js-n52gtt?file=index.js
import * as ulid from 'ulid'; import { CrockfordBase32 } from 'crockford-base32'; function uuidBuffer(buffer) { if (buffer.length != 16) throw new Error("Invalid buffer length for uuid: " + buffer.length); if (buffer.equals(Buffer.alloc(16))) return null; // If buffer is all zeros, return null var str = buffer.toString('hex'); return str.slice(0, 8) + "-" + str.slice(8, 12) + "-" + str.slice(12, 16) + "-" + str.slice(16, 20) + "-" + str.slice(20); } const ulidString = ulid.monotonic(); console.log("ulid", ulidString) const buffer = CrockfordBase32.decode(ulidString); console.log('decoded', new Uint8Array(buffer).toString()); const res = CrockfordBase32.encode(buffer); console.log('encoded', res); console.log("uuid", uuidBuffer(buffer))
This just fails with
ulid.monotonic is not a function
. It does haveulid.monotonicFactory
and if I use that, then it works just fine. I tried many times.Maybe just use ulidx?
thank you, let me check again!
UUID is more productive data format in db, so we need to have ulid as uuid
Cold you add a method to get ulid as uuid, please?