Feedback on collectable

[nickray]

I'm coming at this from the perspective of having need for an allocation-free X509 certificate generator, or more generally, the "DER encoder to end the plethora of semi-done DER encoders" :)

The feedback is that collectable (or a related trait for random insert e.g insertable or spliceable) should should have some kind of "(overlapping) memmove" trait (and not just an Vec::insert-style operation that is called repeatedly). std::Vec has splice, which is (maybe still?) slow: https://internals.rust-lang.org/t/add-vec-insert-slice-at-to-insert-the-content-of-a-slice-at-an-arbitrary-index/11008. In heapless-bytes, I implemented an insert_slice_at by hand: https://docs.rs/heapless-bytes/0.1.1/src/heapless_bytes/lib.rs.html#141-156.

A need arises in allocation-free TLV encoding (to know L, need to encode V or somehow know its encoding's length first - both in the same pre-allocated buffer). I think der::Encodable solves this kind-of at compile time, by requiring the encoded_length method in the trait, which someone has to implement on composite types (at least I think that's what happens - do correct me if I'm wrong). So you could easily do a to_heapless_vec implementation there; I assume part of the motivation for collectable is to abstract over this and get a to_collectable method.

Whereas derp::Der::nested (which I think is an extraction from ring) and x509::der::der_tlv both do an allocation. I fix this for my current use cases in asn1derpy::Der::nested (a temporary fork of derp) exactly by doing an insert_slice_at.

But I'd really like a shared combinator-style approach (like x509) without the allocations. Ideally then der would grow to cover both fixed types and dynamic encoding. Given both these libraries have prestigiously short names :)

The goal for me is an API where the generic n: usize capacity parameter (or N: ArrayLength<u8> en attendant min_const_generics) only needs to be specified once in the return type. Saying this from brute-forcing a heapless implementation of x509.rs in https://github.com/nickray/x509.rs/blob/heapless/src/der.rs#L107-L116, which has multiple issues:

• explicit heapless::ArrayLength bounds all over the place (which I'd hope that a more developed collectable could avoid)
• explicit type hints about the N everywhere
• the mentioned allocations in x509's der_tlv implementation (which could be replaced by asn1derpy's mem-move, using collectable, if this had some kind of insert_iterator_at / efficient splice.

What do you think? :)

[tarcieri]

I'm coming at this from the perspective of having need for an allocation-free X509 certificate generator, or more generally, the "DER encoder to end the plethora of semi-done DER encoders" :)

I'd really like to have this as well. Have you checked out the der crate? I've been doing quite a bit of work on it lately...

[nickray]

yup, comparing der and x509 with my own stuff is where this originates

[tarcieri]

I think der::Encodable solves this kind-of at compile time, by requiring the encoded_length method in the trait, which someone has to implement on composite types (at least I think that's what happens - do correct me if I'm wrong).

Composite types don't actually have to implement Encodable. Rather, there's a higher-level Message trait which composite types can use so long as their fields are Encodable, and a blanket impl of Encodable for Message which constructs a sequence.

The API is callback-based to allow construction of intermediate field encoder types if necessary. Otherwise composite types only need to construct a slice of trait objects which represent their fields.