Porting picoquic to network simulators

[clarkzjw]

People in the network research community are interested in getting a reliable and current QUIC module for ns-3, https://groups.google.com/g/ns-3-users/search?q=quic.

There was some effort before: https://github.com/signetlabdei/quic (and https://github.com/signetlabdei/quic-ns-3 is the in-tree implementation fork of ns-3) is probably the most widely used one and is available on the ns-3 contributed "app store", https://apps.nsnam.org/app/quic/.

Some recent attempts to create external bindings/wrappers to existing QUIC implementations, such as https://github.com/SoonyangZhang/quic-on-ns3, use a fork of Google's quiche.

Although there are many QUIC implementations that align with the IETF RFCs and have participated in QUIC interop testing, the QUIC module in ns-3 did not fully implement the QUIC specifications and lacks active maintainers in recent years. Consequently, it can't directly interoperate with other IETF QUIC implementations such as Picoquic or xquic through fd-net-device like what the existing ns-3 TCP/IP stack can do. Given the ongoing discussions regarding future QUIC features and other extensions like multipath QUIC and others, it seems like the current ns-3 QUIC module might fall further behind.

I recently talked to several people in the ns-3 developer community that they are indeed interested in creating a new QUIC module for ns-3. Several different approaches we talked about are:

• Developing ns-3 bindings or wrappers for established external QUIC implementations, such as Picoquic
• Porting an existing QUIC implementation to ns-3, similar to TCP/IP stack in ns-2.

Creating bindings or wrappers for external QUIC implementations offers flexibility – given that QUIC operates in user space, it paves the way for future researchers to integrate varied QUIC implementations with ns-3, as long as a set of stable northbound ns-3 QUIC APIs is well defined. This would be useful for tasks such as evaluating congestion control implementations in different QUIC libraries.

However, back-porting an existing QUIC implementation to ns-3 allows us to tailor the library to ns-3's specific needs, such as options to disable encryption in simulations. This, though, presents more challenges in bringing in upstream updates to ns-3, which could lead to significant deviation from the original implementation over time.

I saw there are some recent developments on Picoquic that are decoupling OpenSSL and making Picoquic more modular. I guess eventually it would be more feasible to port Picoquic to ns-3.

The goal of this potential work is that QUIC should eventually be available in the ns-3 core distribution, just like TCP/IP, is able to interoperate with QUIC on real networking stacks and be maintainable for developers, and easy to use for users.

What's your general opinion about this? Our preliminary discussions lean towards porting Picoquic to ns-3 though.

[huitema]

My priority would be to avoid a fork. This already happened once with the "pquic" effort, and as could be expected the pquic fork diverged.

We could certainly develop an ns-3 adapter of some kind -- maybe an alternative to the socket adapter. If we want it maintained, we will need to develop functional tests using ns-3, to verify that new functionality first does not break the ns-3 compatibility, and second is available from ns-3.

We should discuss packaging. Maybe there is a way to use picoquic as a component in ns-3. The Quirq prototype developed with Cisco does that: the application's cmake just pulls picoquic and its dependencies.

Any other issue?

[clarkzjw]

This already happened once with the "pquic" effort, and as could be expected the pquic fork diverged.

That's a fair point.

Maybe there is a way to use picoquic as a component in ns-3. The Quirq prototype developed with Cisco does that: the application's cmake just pulls picoquic and its dependencies.

ns-3's build system recently also moved to cmake, so this won't be a big issue I guess.

Ideally, we could use picoquic as a library and only use it's public APIs through picoquic.h.

[huitema]

I did not know that ns-3 used cmake. That makes integration much simpler!

Do we have a good idea of the requirements ? I suspect that simulations should be able to specify parameters, addresses, etc. Also, I suppose we would want some code to send/receive packets through/from simulated networks.

[clarkzjw]

I suppose ns-3 QUIC might have to work under two "modes":

1. Pure network simulation within ns-3 In this case, people might want to disable encryption to simplify certain things, either for evaluating encryption overhead or other potential use cases. Encryption is just one example, there might be others.

2. Interoperate with real network

I suppose we would want some code to send/receive packets through/from simulated networks.

In this case, ns-3 has fd-net-device module, example of ping: https://github.com/nsnam/ns-3-dev-git/blob/master/src/fd-net-device/examples/fd-emu-ping.cc

// Allow ns-3 to ping a real host somewhere, using emulation mode
//
//   +--------------------------+
//   |           host           |
//   +--------------------------+
//   |      ns-3 simulation     |
//   +--------------------------+
//   |         ns-3 Node        |
//   |  +--------------------+  |
//   |  |      ns-3 TCP      |  |
//   |  +--------------------+  |
//   |  |      ns-3 IPv4     |  |
//   |  +--------------------+  |
//   |  |   FdNetDevice or   |  |
//   |  | NetmapNetDevice or |  |
//   |  |    DpdkNetDevice   |  |
//   |--+--------------------+--+
//   |         | eth0 |         |
//   |         +------+         |
//   |            |             |
//   +------------|-------------+
//                |
//                |       +---------+
//                .-------| GW host |--- (Internet) -----
//                        +---------+

In this case, ns-3 QUIC will need all the necessary QUIC capabilities, such as encryption, version negotiation etc. to properly interact with Picoquic, for example.

[mpiraux]

For what it's worth, we integrated our fork with ns-3 using the Direct Code Execution framework. It enables binaries to run within an ns-3 simulation. I believe it is not as flexible as an ns-3 model, given that simulations interact with the QUIC implementation binary, but for our purposes of testing the code we developed, it proved useful. It might be one more approach to consider but it has different implications than what is suggested above as it is not an ns-3 model.

The code is at p-quic/pquic-ns3. We essentially adapted the quic-network-simulator to work in pure simulation with DCE. I will happily provide more details on this if needed.

[clarkzjw]

@mpiraux My main concern about DCE is that it introduces another layer of complexity. As [1] and its slides [2] mentioned, keeping the compatibility of DCE with new Linux kernel releases is also a big hurdle.

An ns-3 native QUIC implementation or at least wrappers to Picoquic would be much easier to use for researchers and easier to maintain for ns-3 developers in the long term. Hopefully, it can become part of the ns-3 core distribution eventually.

[1] Parth Pratim Chatterjee and Thomas R. Henderson. 2022. Revitalizing ns-3’s Direct Code Execution. In Proceedings of the 2022 Workshop on ns-3 (WNS3 '22). Association for Computing Machinery, New York, NY, USA, 57–64. https://doi.org/10.1145/3532577.3532606 [2] https://www.nsnam.org/workshops/wns3-2022/08-chatterjee-slides.pdf

[kaloyan-ignatov]

Our research team is also interested in having a working QUIC implementation in ns-3. Our efforts are currently centered around figuring out whether bringing the native ns-3 QUIC implementation (https://github.com/signetlabdei/quic-ns-3) up to speed with RFC 9000 would be a feasible solution.

We have already fixed some issues regarding the compatibility of the model with ns-3 3.37, but we are looking to explore further solutions, as although the most desirable, a native implementation would take a lot of time and resources. DCE is what we deliberated on using next, but the problems mentioned by @clarkzjw would present an additional challenge, on top of the limited access to simulation data.

The most balanced solution for now seems to be to implement bindings for picoquic.