Talk about interface

[dinosaure]

Idea behind buffet is to provide a safe interface to manipulate buffer (string, bytes and bigstring). buffet has different levels about interface:

• Buffet0 this is the foundation for the next level, it's just a bunch of external, bigstringaf and nothing else. We provide getter and setter on char, int, int32 and int64 (little-endian and big-endian) like what bigstringaf does already.
• Buffet1: we start with a GADT to keep kind of buffer. We merge all on a type 'kind tag type which is used to dispatch functions to the good implementation. Buffet1.get bytes will call Bytes.get.
• Buffet2: we continue with a GADT but put some information like if we have a read, write or async (leave the global lock) capability on one of kinds of buffer. Limitation on GADT can not permit us to use a sub-typing (like we do in a common way with polymorphic variant). In this implementation, we lint some pattern-matching like the dispatcher on the set/unsafe_set function which don't allow a String kind for example.
• Buffet3: we put a new information, an offset and a length with type ('a, 'k) Buffet1.access which check at any step if slice is available on the buffer 'k. I decided to return a result type, Ok if we have an access on the buffer 'k with a specific slice or an error, you can see the function commit or with_off and with_len with the bind operator >>=.
• Buffet4: again, a GADT to hidden/pack type ('a, 'k) Buffet3.meta with 'k. Then, we check with GADT if we are allowed to read/write and so on on it with reflection. In other case, we raise an exception.

I'm currently on a new implementation of Buffet2 which use polymorphic variant. Then, we need to do some benchmark to compare them and see which is the best:

• discard String case
• or raise an exception

If you have any advise, I will happy to discuss about that (@hannesm, @samoht, @cfcs, @let-def, @Drup).

[Drup]

What's the goal here ? Do you really want to be able to get values that can be dynamically be string/bytes/bigstrings, or do you want to be able to write your encoding/decoding algorithms on various structures easily ?

The current version is ridiculously full of indirections, to the point where you will seriously hurt performances. Using module abstractions, with various signatures that correspond to the various features, would allow you to keep indirections to a minimum.

[dinosaure]

I would like to have an abstraction between bigstring/string and bytes and choose which one I use it dynamically. This is the main purpose of this library where sometimes I would like to use string instead bigstring.

I agree about indirection and the bottleneck is about the dispatch between kind of buffers.

It seems interesting to provide Buffet3 feature (slice) directly over Buffet0 and implement capabilities with polymorphic variants over Buffet0 too.

I will try to provide signatures for each feature and see what happen.

[dinosaure]

We can more precisely talk about performances issues with benchmark/benchmark.ml now and bechamel. I will make some tests step by step and print results here.