AccelerateWithOptane / lab

Request access to Optane powered bare metal infrastructure for performance-testing and analysis purposes
15 stars 3 forks source link

Proposal: NATS Streaming #2

Open ColinSullivan1 opened 6 years ago

ColinSullivan1 commented 6 years ago

Name, email, company, job title

Colin Sullivan, colin at nats.io, Synadia Ivan Koslovic, ivan at nats.io, Synadia

Thank you for the invitation! If possible, the 2nd week of June (7/11/2018) would be ideal for us.

NATS

NATS is a seven year old, production proven, cloud-native messaging system - an infrastructure to send and receive messages between distributed systems. The core principles underlying NATS are performance, scalability, and ease-of-use. Based on these principles, NATS is designed around the following core features:

NATS is a hosted project within the Cloud Native Computing Foundation

How does the open source community benefit from your work?

NATS is being deployed globally by thousands of users and companies, spanning use-cases including mobile applications, microservices and cloud native applications, and IoT.

Is the code that you’re going to run 100% open source? If so, what is the URL or URLs where it is located?

NATS servers and client libraries are 100% open source, Apache 2.0 licensed, found under the nats.io Github organization. Specifically for testing here we plan on using code from these repositories: https://github.com/nats-io/gnatsd https://github.com/nats-io/go-nats https://github.com/nats-io/nats-streaming-server https://github.com/nats-io/go-nats-streaming https://github.com/nats-io/nuid

We may develop some specific tests for this program, which we commit to open source as well.

What infrastructure (computing resources and network access) do you need? (see: https://www.packet.net/bare-metal/)?

After disk I/O, NATS streaming benefits from a fast CPU and high network bandwidth capabilities. Something equivalent to c1.xlarge.x86 for Intel Optane machines would likely achieve the best performance.

Let us know if you need short-term (one time) support, or if this is a request for continuous ongoing support. If possible, please identify foundations or other support organizations that can help with long-running projects.

We aren't anticipating any real need for support but would certainly like guidance for the duration of the test program.

What performance-focused articles has your project published?

Is your project intensely interested in performance, especially where disk I/O is concerned? Have you written about it, or shared results of testing? Share anything that shows your focus.

We are, and have some third party articles, blog entries, and talks. Here are a few: https://dzone.com/articles/benchmarking-nats-streaming-and-apache-kafka https://nats.io/blog/nats-c-perf-analysis/ https://nats.io/blog/strange-loop-recap/

Please state your contributions to the open source community and any other relevant initiatives

NATS can be found in many software projects and products, and we hope to help even more projects as a CNCF project. We have a strong ecosystem generated around NATS with an active slack community, and many user contributions. We are dedicated to expanding and improving the NATS project feature set to meet our users' needs and that of the NATS ecosystem.

Would you be willing to share your analysis and results publicly?

We are interested in blog posts, meetups and conference presentations. AccelerateWithOptane would be more than happy to host your blog posts or link to them, and may coordinate performance-oriented meetups and conferences. Are you open to sharing?

Absolutely!

Are you interested in testing Optane SSDs with Intel Memory Drive Technology (IMDT)?

IMDT extends system memory transparently by integrating Optane SSD capacity into the memory subsystem, effectively expanding your memory pool beyond system DRAM capacities at a much lower cost. Check here for more information.

Yes, we would - sounds very interesting. We have a memory storage mode in NATS streaming that could benefit greatly from this.

deitch commented 6 years ago

Welcome, @ColinSullivan1!

If possible, the 2nd week of June

Done. I have you marked for that week.

Yes, we would - sounds very interesting. We have a memory storage mode in NATS streaming that could benefit greatly from this

Great. Let's start with the basic test as a block device, and we will have one of the great Intel engineers enable your server for IMDT when ready. There is a licensing process involved (but no charge to you! :-) ).

If you have not done so already, please sign up for Packet Slack if you have not already, and mark your user ID here, so I can invite you to the specialized Optane support channel (in addition to the usual community channel, of course). I can be reached under the ID deitcher.

Also, if you not done so already, please sign up for a free Packet account here. It is our primary support portal, and where we will make the servers available.

Welcome!