search

haxetink / tink_io

Asynchronous I/O everywhere
https://haxetink.github.io/tink_io
MIT License
18 stars 12 forks source link

API changes. #11

Closed back2dos closed 6 years ago

back2dos commented 8 years ago

Expanding on #7 I am contemplating some slightly "drastic" API changes.

The pooled circular buffers are really nice and they help keeping the memory usage very low, but the more I move forward, the more I feel like this is an implementation detail that the user should not have to be bothered with.

So I am proposing the following simplifications:

  1. The Sink will have a write method defined like so: function write(bytes:Bytes, start:Int, length:Int):Surprise<Progress, Error>;. The reason for this is that it's not so easy to get at the data in a buffer. I'm also not 100% sure the current buffer is the best approach. The linux kernel seems to use a circular buffer of chunks (one advantage here is that the producer can write to one chunk while the consumer reads from another). Java's nio also does something different (although I haven't quite understood how it actually works^^). Going down an abstraction layer here allows to try different approaches.
  2. The read method will be removed from Source and what will remain is the pipeTo method. This greatly reduces the requirements on sinks. Noone gets to read from a sink directly. If you want data, you'll have to ask the source to pipe it to you. So for example parsers will always be sinks. The type of reading there is now can still be achieved by having a Sink that accumulates some data and then allows you to read it out the way it is now.

This is no small change, but I think it will be worth while.

Any comments @kevinresol @benmerckx ?

benmerckx commented 8 years ago

My knowledge of sources and sinks is limited to the small amount of time I've looked at the tink_io sources and by using them. I have zero experience with these in other languages/platforms so I can't really provide much feedback.

I can't remember using read at all by the way, so I think having only pipeTo will do.

Also if you have anything you could link to which could make me understand the usage and design choices better I'd appreciate it :)

kevinresol commented 8 years ago

Same here. Async io is something new to me. Even I use nodejs I have barely any knowledge about read/writestream.

kevinresol commented 6 years ago

Is this already done?

back2dos commented 6 years ago

Yes and no. Let's just close this ;)