Chapter 2 | Figure 2-3 | Proposal to introduce evolution of IO models and their pattern on Throughput vs Latency

[mpalriwal-Netflix]

BookLink

To enhance the understanding of the relationship between throughput and response time, it would be beneficial to consider the evolution of I/O models: Blocking I/O (BIO), Non-blocking I/O (NIO), and Asynchronous I/O (AIO).

• Blocking I/O (BIO): Typically results in higher response times as throughput increases due to its sequential processing nature, leading to significant queueing delays.
• Non-blocking I/O (NIO): Offers moderate improvements by allowing other operations to proceed while waiting for I/O, reducing queueing delays compared to BIO.
• Asynchronous I/O (AIO): Provides the most efficient handling of high loads, as it processes I/O operations in the background, minimizing response time even as throughput approaches maximum capacity.

Incorporating these models into the discussion can provide a clearer picture of how different I/O strategies impact system performance under varying loads.

The same is being discussed in one of my InfoQ article https://www.infoq.com/articles/reactive-java-vertx-deep-dive/

Blocking I/O (BIO): the steepest curve. Non-blocking I/O (NIO): the moderately rising curve. Asynchronous I/O (AIO): near the flattest curve.

[ept]

I'm unclear about the distinction between non-blocking and asynchronous I/O – in my mind these two are equivalent (the calling thread does not wait for I/O to complete, but rather picks up the response later via an event loop that perhaps dispatches a callback). The article you link to doesn't really explain the distinction either. Could you elaborate?

[mpalriwal-Netflix]

Thank you for your feedback, Martin. I understand the confusion, as non-blocking and asynchronous I/O are often used interchangeably, but there are subtle differences captured here in an IBM recent blog :

• Non-blocking or New I/O (NIO): In this model, I/O operations return immediately without waiting for data. The calling thread can continue executing other tasks and periodically check if the I/O operation is complete. This architecture encompasses two main components: Buffer & Channel.

• Asynchronous I/O (AIO): AIO extends NIO's capabilities by introducing an asynchronous, non-blocking model that uses callbacks to handle operations. . The calling thread doesn't need to check the status, as the system handles the completion notification. The central object in asynchronous I/O is called the Selector. A Selector is where you register your interest in various I/O events

Key Differences

• Blocking: NIO can be non-blocking, but AIO is inherently asynchronous, meaning it doesn't block at all.
• Notification: NIO may require polling or checking for completion, while AIO uses callbacks or events for notification.
• Complexity: AIO can be more complex to implement due to its asynchronous nature but offers greater efficiency for high-load scenarios.

I hope this clarifies the distinction. If you have further questions, I'd be happy to elaborate.

---

[mpalriwal-Netflix]

To add a recent news around this topic, similar to a multi-reactor pattern I've shared here

With intelligent multi-core utilization and asynchronous I/O threading boost throughput to 1.2 million requests per second, tripling the performance of previous versions. By switching from Redis's archaic single-thread event loop threading model to a sophisticated, I/O operations multithreaded approach, Valkey has vastly increased its speed.