Add support for variable length members

[Dekker1]

I've been having a lot of fun playing with the bilge library. Thank you for writing it! Although it solves most of the bit-fiddling problems for which I would have used C(++). The remaining trick that I'm really missing in Rust is to allocate a number of elements that is known at allocation time in an object, so variable length field times.

Optimising the allocation for variable length member objects often goes hand in hand with other bit-specifying problem. For example, I'm looking at a problem where I have a object that has a header that can fit in 24 bits, and then a few, n < 255 32-bit objects. Since the length can be stored in 8 bits, I would like to allocate a single object of 32+n*32 bits.

Currently the only crate that will allow you to do this seems to be varlen. Although it is a good step, it seems that it has some problems. Most importantly, it does not seem to allow you use size type for an array that is smaller than the types contained in the array. Although it allows me to specify Array<u32,u8> (in addition to my u24), the size layout method shows that it allocated 2*32+n*32. So I would love to see something that is easy to use in combination with this library. Especially, if it can make the specification and usage of these types easier, as it does for the other bit-operations.

[hecatia-elegua]

I've been a bit sick lately, sorry for the late response. It sounds like deku can do something like this, would one of those, or some of their other attributes help? https://github.com/sharksforarms/deku/blob/d04559605bed4476b1a78aaffd843377d6ab8fe6/src/attributes.rs#L298 https://github.com/sharksforarms/deku/blob/d04559605bed4476b1a78aaffd843377d6ab8fe6/src/attributes.rs#L333

I'll have to look into it later.

[Dekker1]

Although I think deku is trying to allow users to read in the type of data that I'm talking about, the allocation of the objects in Rust is using Vec. This means that the members are located in a different object, not with the header.

Instead of

struct A {
    header: u24,
    count: u8,
    items: Vec<u32>,
}

I would like to have a type similar to

struct B {
    header: u24,
    count: u8,
    items: [u32],
}

But importantly, the number of members should be kept only in the object as count, so that every reference to B does not needs an additional 64 bits to keep the size of the object. (Which is what normally happens for unsized objects in Rust).

[hecatia-elegua]

Did https://github.com/reinerp/varlen-rs/issues/3 solve your issue fully?

[Dekker1]

It did indeed solve my problem. This means I can now indeed use these two libraries together, although I think I would still love to see if the clarity that this library brings could be translated into support for variable length types. Thanks at least for checking in :)