jonsequitur
commented
1 week ago I don't know that the number of kernels in and of itself would be an issue but we've never done any perf tests on this kind of scenario.
The first thing that comes to mind though is the scheduling model. Under a single composite kernel, all subkernels share a scheduler by default, and commands are queued so that parallelism isn't introduced. You can of course write code that goes parallel. So the default scheduling behavior would be a by-design bottleneck for your scenario.
But this behavior can be altered. Here are a few approaches that can work today:
- It would be possible to write a kernel that offloads commands to its own queue (or for example the default task scheduler).
- ...or a middleware that does this for all kernels.
With a few modifications, the following would also be possible:
- Allow the scheduler to be replaced per subkernel so they're independent of one another. (This API is currently internal.)
TLDR: Is there a limit to the number of kernels in a composite kernel? Is managing a large number of them scalable?
While waiting for good Wasm usability (multithreading support, large load of all necessary files, etc.) to use .NET Interactive in Client-Blazor, I'm switching my experiments to Server-Blazor.
My use case: The user has access to plenty of common/predefined variables available immediately for all users and then runs their own code. Users also have access to their specific preloaded variables available only to them.
For example, I have data for the London area and subway stations available to everyone, and private places (current job, school) available only to specific users.
I wonder if I could create a copy of an (F#) kernel for each user by adding them to a composite kernel. Such a copy could receive (via share) access to common variables and receive their private data for their use. I would remove these kernels after some time when the users stop executing them.
I am aware there is a potential for vulnerability/malicious code that I should protect against, but that is not part of this question. I am seeking the simplest possible scalable pattern to allow a number of users to share one (theoretically powerful, with a lot of CPUs and memory) app service. I believe I don't want something similar to GitHub Codespaces, where a user may "provision" their own environment.
I also know that the F# kernel locks internal access to a script engine, and even with a powerful server, it may not be super scalable. However, having dozens of parallel (64-128) interactive engines should be sufficient. Is a kernel/composite a lightweight enough structure for that purpose?
@jonsequitur @colombod