atilaneves / cerealed

Powerful binary serialisation library for D
BSD 3-Clause "New" or "Revised" License
92 stars 3 forks source link

Max size for serialization is 64KiB #11

Closed aermicioi closed 8 years ago

aermicioi commented 8 years ago

Why does Cerealed limit max size of serialized data to 64KiB?

Consider:

import std.stdio;

import cerealed;

struct ubyted {
    ubyte[] payload;
}
void main() {
    ubyte[] buff = new ubyte[2 ^^ 16];
    Cerealiser cer;

    cer ~= buff;
}

This code will fail with assert error on line 87 in cereal.d Error: cerealed-0.6.6/src/cerealed/cereal.d(87): overflow

A buffer with length less with one will not trigger assert. Is it by design, or is it a bug?

atilaneves commented 8 years ago

I was wondering when someone would notice :P It's a ushort by default, but it can be a larger integer. I hadn't made it easy to select a different one because I didn't need to and in the transition from classes to structs I kind of hid the functionality by accident. Fix coming up.

atilaneves commented 8 years ago

In your example now:

import cerealed;

struct ubyted {
    ubyte[] payload;
}
void main() {
    ubyte[] buff = new ubyte[2 ^^ 16];
    Cerealiser cer;
    cer.grain!uint(buff);
}

If the array is in a struct, tag it with @LengthType:

    struct Foo {
        @LengthType!ubyte ushort[] arr;
    }

    auto enc = Cerealiser();
    auto foo = Foo([7, 8, 9]);
    enc ~= foo;
    assert(enc.bytes == [3, 0, 7, 0, 8, 0, 9]);
    auto dec = Decerealiser(enc.bytes);
    assert(dec.value!Foo == foo)