andreiavrammsd / cpp-channel

Thread-safe container for sharing data between threads
https://blog.andreiavram.ro/cpp-channel-thread-safe-container-share-data-threads/
MIT License
413 stars 30 forks source link

Batch operations? #31

Open Elgot opened 1 year ago

Elgot commented 1 year ago

Would it be possible to support batch reads/writes in some form? If you want to write all elements in a container to a channel it seems a bit inefficient to acquire the lock for every item. This would be nice:

msd::channel<int> chan{10};
std::vector<int> test {1, 2, 3, 4, 5};
std::array<int, 2> test2;
chan << test;

int a, b, c;
chan >> a; //1
chan >> b; //2
chan >> test2; //{3, 4}
chan >> c; //5
andreiavrammsd commented 1 year ago

Do you have performance issues? What is your specific use case?

To make this feature generic, it's needed to write any range of elements from a container. Thus an iterator would be used:

msd::channel<int> chan{10};
std::vector<int> test {1, 2, 3, 4, 5};

// write some of the elements
chan.from_iterator(test.cbegin(), std::next(test.cbegin(), 2));

// or write them all
chan.from_iterator(test.cbegin(), test.cend());

// then read
std::array<int, 2> test2;
std::size_t how_many = 3;
chan.to_iterator(test2.begin(), how_many);

A syntax like

chan << test;
chan << test | std::ranges::views::take(2);

that supports any range of the container would be possible only in C++20, and I want the channel to be compatible with C++11.

Another thing to consider is what happens if you want to read more elements that currently are in the channel. It should block the reading and wait until there are enough elements. What happens if you want to read more than the channel's capacity? Maybe it could be considered undefined behavior...

I see this as a challenging feature and I would take my time on it. Any input on this is welcome.

Elgot commented 1 year ago

Do you have performance issues? What is your specific use case?

I'm not sure, but this kind of class could be very handy in many situations, and it should be as efficient as possible. Perhaps you could use it as a buffer between two components in a signal processing chain where one block produces samples of length N and another expects length M? The blocking would be nice to add back pressure too.

A syntax like

chan << test;
chan << test | std::ranges::views::take(2);

that supports any range of the container would be possible only in C++20, and I want the channel to be compatible with C++11.

Perhaps the modern syntax could be optional (or enabled only for c++20 or later)?

Another thing to consider is what happens if you want to read more elements that currently are in the channel. It should block the reading and wait until there are enough elements.

I would expect it to block (like when you read a single element). You might want a way to read all available elements too though.

What happens if you want to read more than the channel's capacity? Maybe it could be considered undefined behavior...

Undefined behaviour is not very nice if it can be avoided. It should probably be handled in some way...

andreiavrammsd commented 1 year ago

Thanks for the input! I will consider the info.

Nevermore1994 commented 1 year ago

@Elgot Hi, you can refer to this repo if you need it. https://github.com/Nevermore1994/Cpp-Channel

Elgot commented 1 year ago

@Elgot Hi, you can refer to this repo if you need it. https://github.com/Nevermore1994/Cpp-Channel

Cool, I’ll have a look!