Closed kjetilkjeka closed 2 years ago
This is mostly based off opinion but also facts, i chose to not expose null streams (well, to keep what rustacuda was doing in not exposing them) due to several reasons:
I believe null streams were a big mistake in CUDA, they were made just to simplify launching kernels, but they hide so many details under the surface that at this point they should be phased out.
I understand this is a difficult decision. If the motivation is to create an opinionated library for using the GPU to accelerate Rust code the approach makes a lot of sense.
To help you understand where I'm currently coming from. We're currently looking into writing one of our applications in Rust. We will need to use computations on the GPU in our application. We have a strong C++ team that is interested in Rust, but is not willing to introduce any technical risk. This means we might use Rust for host code and Cuda C/C++ compiled to ptx on device.
Due to the early state of Rust GPU it makes sense to, for now, learn from the patterns as we use in C++. This means we need to be able to reach for all/most parts of the cuda runtime library. I'm not sure if there's any way to unify our concerns? If not I think it might be a place in the ecosystem for a non-opinionated library attempting to expose most Cuda functionality as well.
Streams are not a concept exclusive to rust-gpu, any good cuda application uses streams because the null stream is bad for performance when you want to run kernels async or multiple kernels. You lose nothing by using explicit streams over null streams. You don't even lose interop capabilities because streams sync with the default stream implicitly. So i don't really understand your concern, explicit streams only gain more things you can do, they don't lose any capabilities. This gets even more complex when you factor in legacy default streams vs per-thread default streams. It is so easy to shoot yourself in the foot unless you are deeply familiar with the details of the driver API.
One of the major goals of this project is clearing away the legacy cuda cruft and moving on to proper ways of doing things. This is exactly why:
atomicAdd
in favor of writing atomics once and doing it correctly using modern ptx instructions.It is critical to establish a standard for how to write CUDA applications idiomatically, and it is my responsibility to establish such a standard at such an early stage. Exposing the default stream right now would encourage people to create subpar code and ignore such a critical part of GPU programming. Therefore i will not expose the null stream, you can emulate them with a lazy static perfectly, but there is rarely a reason to do this.
Consider this, would you say the same if Vulkan had a single blocking universal queue instead of real queues?
I understand your motivation and think it is an excellent goal. As a long time goal we are also very interested in finding out the most idiomatic way to use Rust in Cuda and move towards it. Even though we might not be able to use everything here due to some of our short deadlines we will still follow everything you do here tightly and hope to cooperate as much as possible.
Are there any reasons for currently not exposing the null stream in cust?
Would there be any problems to implementing
Default::default()
forStream
as the null stream?A lot of traditional C++ cuda code uses the null stream. I think exposing it in rust as well can lower the complexity of having to manage streams yourself. It can also make the on-boarding experience close to what it is in C++.