Compactr

Schema based serialization made easy

What is this and why does it matter?

Protocol Buffers are awesome. Having schemas to deflate and inflate data while maintaining some kind of validation is a great concept. Compactr's goal is to build on that to better suit the Javascript ecosystem.

    npm install compactr

Implementation


const Compactr = require('compactr');

// Defining a schema
const userSchema = Compactr.schema({ 
    id: { type: 'number' },
    name: { type: 'string' }
});

// Encoding
userSchema.write({ id: 123, name: 'John' });

// Get the header bytes
const header = userSchema.headerBuffer();

// Get the content bytes 
const partial = userSchema.contentBuffer();

// Get the full payload (header + content bytes)
const buffer = userSchema.buffer();

// Decoding a full payload
const content = userSchema.read(buffer);

// Decoding a partial payload (content)
const content = userSchema.readContent(partial);

Size comparison

JSON: {"id":123,"name":"John"}: 24 bytes

Compactr (full): <Buffer 02 00 01 01 04 7b 4a 6f 68 6e>: 10 bytes

Compactr (partial): <Buffer 7b 4a 6f 68 6e>: 5 bytes

Protocol details

Data types

Type	Count bytes	Byte size
boolean	0	1
number	0	8
int8	0	1
int16	0	2
int32	0	4
double	0	8
string	1	2/char
char8	1	1/char
char16	1	2/char
char32	1	4/char
array	1	(x)/entry
object	1	(x)
unsigned	0	8
unsigned8	0	1
unsigned16	0	2
unsigned32	0	4

Count bytes range can be specified per-item in the schema*

See the full Compactr protocol

Benchmarks

[Array] JSON x 737 ops/sec ±1.04% (91 runs sampled)
[Array] Compactr x 536 ops/sec ±1.25% (82 runs sampled)
[Array] size: { json: 0, compactr: 10 }

[Boolean] JSON x 996 ops/sec ±0.84% (93 runs sampled)
[Boolean] Compactr x 1,439 ops/sec ±1.24% (83 runs sampled)
[Boolean] Protobuf x 2,585 ops/sec ±1.32% (91 runs sampled)
[Boolean] size: { json: 23, compactr: 5, protobuf: 5 }

[Float] JSON x 713 ops/sec ±1.29% (88 runs sampled)
[Float] Compactr x 1,056 ops/sec ±1.38% (86 runs sampled)
[Float] Protobuf x 2,432 ops/sec ±1.45% (88 runs sampled)
[Float] size: { json: 41, compactr: 12, protobuf: 12 }

[Integer] JSON x 954 ops/sec ±0.77% (93 runs sampled)
[Integer] Compactr x 1,520 ops/sec ±1.21% (89 runs sampled)
[Integer] Protobuf x 2,676 ops/sec ±1.22% (90 runs sampled)
[Integer] size: { json: 24, compactr: 8, protobuf: 7 }

[Object] JSON x 512 ops/sec ±0.62% (90 runs sampled)
[Object] Compactr x 412 ops/sec ±1.46% (81 runs sampled)
[Object] Protobuf x 898 ops/sec ±0.92% (93 runs sampled)
[Object] size: { json: 46, compactr: 13, protobuf: 25 }

[String] JSON x 432 ops/sec ±0.79% (88 runs sampled)
[String] Compactr x 428 ops/sec ±0.87% (87 runs sampled)
[String] Protobuf x 738 ops/sec ±0.71% (90 runs sampled)
[String] size: { json: 57, compactr: 14, protobuf: 28 }

Want to help ?

You are awesome! Open an issue on this project, identifying the feature that you want to tackle and we'll take the discussion there!

compactr / compactr.js

readme