DTolm
commented
4 years ago Hello, I am Dmitrii, the creator of VkFFT and Vulkan version of Spirit. I saw your comment and I believe your project is the way to go, if Vulkan wants to be popular in compute or, specifically, scientific field. There has to be some kind of a layer that will move them as further as possible from the way I developed Vulkan Spirit. There are some important things, that have to be clarified in the absolute beginning, which are related to the architectural problems and how this layer should be designed. These things are based on my experience and I have big faith in them, though some people may disagree and they have the right to do so. I. Target audience.
- Unless the user understands how the GPU works, the code will most certainly be bad, no matter if it is Vulkan or CUDA. The amount of information, such as threads, blocks, memory strides, shared memory bank conflicts, register spilling and how to mitigate the negative effects of them is enormous. The user who is not willing to do research will stick to CUDA, I can almost guarantee you. Thus aiming at them is pointless and their code will be slow and unusable anyway. This doesn't apply to users who are only willing to use third-party frameworks based on good code, like tensorflow. These people can, however, be also omitted, as they don't really interact with either Vulkan or CUDA directly.
- People who understand the GPU or willing to do so. The gap between groups can be huge but people with considerable CUDA experience can easily switch to Vulkan. These people will aim at the best possible performance, so they will always doubt this layer and consider to switch to pure Vulkan, if it doesn't perform in the best possible way (this is how I would feel and why I haven't chosen sth like MLIR). Luckily, I know how this can be avoided. II. How I believe the layer should work
- You can't prevent people from writing shader code. The amount of memory transferred from VRAM to the chip during an iteration/command buffer execution is the main limiting factor on 99% of GPU compute tasks (in VkFFT memory transfers take 80% of compute time). So, your solution of making a shader - C-command binding will always be not perfect, as the command most likely can be merged with the previous one and performed while data is still on chip. There are two solutions to this. a)Create a collection of Vulkan simple shader primitives (reduce, scan) and just publish the code, so people can use and modify them as they want. If the code is only 1-2 shader long, they should be able to copy the code to their own shaders, as it will reduce memory transfers. This is hard for user, but will yield the best possible performance in the end. Take a look at how VP, LBFGS and other solvers are created in Vulkan Spirit (see Vulkan_Compute.hpp). Each reduce command is merged with the last compute call there and all commands are merged between reduce calls. The boilerplate of creating them, though, is the thing that should be avoided in the layer in some form. b)Create a C/C++ to glsl command converter script, but, in my opinion, this is next to impossible to do in a way so it doesn't limit pure shader functionality. But it is more user-friendly.
- If there are any big libraries created, their inclusion should be done as an apply command, which adds pipeline dispatches to the user's command buffer. The difference between the library and the primitive is that the user doesn't have to understand the details of how the library works, while the primitive is fairly simple(see: reduce, scan). These libraries should still be accessible according to 1a, though their code may be self-contained and not copied like the collection 1a (but it can still be modified inside the library). Take a look at VkFFT. You call VkFFTAppend and it adds VkFFT to your command buffer. This is almost the same way as cuFFT works. Adding native zeropadding and being able to access every axis and command space between them is what makes it unique and why Vulkan Spirit is up to 3x faster than mumax3. Making the interface as an apply will also reduce command buffer creation/call overhead.
- Aim at zero GPU on CPU dependency during the execution. This is done in Vulkan Spirit, as it doesn't use the CPU after command buffer creation for anything, but asynchronous data saves from the GPU. GPU is connected through PCI-E 3/4 bus which has only 15/30 GB/s of bandwidth. This is at least 20 times lower than GPU - on-chip memory transfer and at least another 20 times lower than on-chip data transfers. Stopping and recreating command buffers takes time and should be avoided. Vulkan Spirit doesn't do that at all during the iteration process. These are the ideas that come right into my mind. Hope they will help, I will expand on them in the future. I am happy to answer questions. Taking a look at VkFFT and Vulkan Spirit will calrify most of the points I made, even though the code is not that clean. Best regards, Tolmachev Dmitrii
axsaucedo
dkgaraujo
alexander-g
unexploredtest
mauvray
ChenKuo
Open issue to openly discuss potential ideas or improvements, whether on documentation, interfaces, examples, bug fixes, etc.