ben-strasser / fast-cpp-csv-parser

fast-cpp-csv-parser
BSD 3-Clause "New" or "Revised" License
2.11k stars 440 forks source link

3 buffers vs double buffer #56

Closed romange closed 6 years ago

romange commented 6 years ago

Hi Ben,

This is not really an issue, just a question. I've read your code (async part) and I noticed that you used 3 buffers: every time you complete the first buffer you copy the second to the beginning, wait for the last third to complete and then you copy it as well . What's the reason for this design? Why not using just double-buffer technique: parse from the first part while asynchronously read into the second half, wait for it to complete and then to parse the second part, asynchronously reading from the first part. It seems simpler so I am guessing there is a catch.

ben-strasser commented 6 years ago

Hi,

lines can span two blocks. The consumer can therefore not fully cosume his block until the producer is finished with the next one. However, the producer cannot finish if he has no new block to move to. We therefore have three blocks.

Best Regards Ben Strasser

On 10/28/2017 02:44 PM, Roman Gershman wrote:

Hi Ben,

This is not really an issue, just a question. I've read your code (async part) and I noticed that you used 3 buffers: every time you complete the first buffer you copy the second to the beginning, wait for the last third to complete and then you copy it as well . What's the reason for this design? Why not using just double-buffer technique: parse from the first part while asynchronously read into the second half, wait for it to complete and then to parse the second part, asynchronously reading from the first part. It seems simpler so I am guessing there is a catch.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ben-strasser/fast-cpp-csv-parser/issues/56, or mute the thread https://github.com/notifications/unsubscribe-auth/ALaAj5crc9WiUYjXnjq6DgsweoMQn1R8ks5swyGUgaJpZM4QJ7gU.

romange commented 6 years ago

Thanks! That explains it :+1: