NetworkBlockDevice / nbd

Network Block Device
GNU General Public License v2.0
450 stars 116 forks source link

Performance optimization #153

Open lordbink opened 11 months ago

lordbink commented 11 months ago

How does nbd manage processes and threads? I am looking into this to find out what optimizations may be required and what settings to watch for as I start to use nbd-server more. Is there anything I need to watch out for when it comes to performance?

Is there a limit to the number of processes that are started? Is there a way to manage the number of processes?

The man page describes that the nbd-server will read requests in a main thread and handle those requests in separate threads. It appears, in my test environment, that a new process is always started when needed, as a client connects.

What is the correct way to get the number of threads?

When I set max_threads to 30, threads always stays at 1. I am checking this with cat /prod/<nbd process id>/status | grep Thread. Does the fact that I am making read-only requests to the server have something to do with this?

My environment

(for context)

Thank you in advance for your help.

L

yoe commented 9 months ago

nbd-server uses glib's GThreadPool to implement async handling of requests. That has a function to set the maximum number of threads, which is what we use when you set max_threads.

At this point, nbd-server still uses fork-per-child semantics, and the threadpool handling is only started after the fork. While I tried improving that situation a few times in the past, unfortunately that resulted in security issues every time, so I'm not going to do that again without a major refactoring of nbd-server first. I still want to do that at some point in the future, but unfortunately I don't have much time to do this right now.

lordbink commented 1 month ago

We are no longer using NBD and switched back to NFS for our use case.

yoe commented 1 month ago

However, the issue still exists, and I do still want to keep it on my radar, so reopening.