aclysma
commented
4 years ago Would skulpin allow such a plugin based system?
There is a very simplistic plugin system and the repo includes a plugin that adds support for imgui, so you could look at that as an example. The plugin system is really nothing more than a swapchain created, swapchain destroyed, and on render callback with some very basic info passed along (like the size and count of the swapchain images).
The way the plugin system for skulpin is set up, you can only draw on top of the skia canvas. The biggest blocker to drawing under the canvas is that the skia canvas is already cleared and would overwrite anything you draw under it.
This library is really targeted towards simple use-cases like prototyping and just getting some basic shapes on the screen. I fully expect many people who start with this library will outgrow it. Really, the plugin system is only there so that imgui can be supported out-of-the-box without adding a required dependency to it. :)
You may be able to do this without using plugins though, some ideas below.
how would integration into Qt/GTK/etc. work?
Ultimately you're integrating with the underlying OS. Qt/GTK etc. will need to expose something like an HWND in windows. This very handy crate (https://github.com/rust-windowing/raw-window-handle) provides a minimal abstract for getting the appropriate OS-level resource. The windowing system (i.e. something like winit, or your own code if you're wrapping Qt/GTK) can produce it and then other crates that need those resources (like this crate or other crates like gfx-hal) can consume it.
Could skulpin expose a raw Vulkan area for plugins to render videos on?
You could consider drawing on the canvas. Two approaches come to mind:
- Use skia's canvas API to blit your image into the canvas. This would be simple but may not meet your performance requirements.
- You could try grabbing the underlying image that the canvas owns and draw to it directly. You may need to find some way to tell skulpin not to clear it. It's likely not much code to do this, but may be tricky if you haven't worked with vulkan before. Please take a look at the existing vulkan/skia interaction code linked below for a hint on how to do it.
The vast majority of the code in skulpin is just setting up vulkan. The actual integration with skia is tiny. The vulkan/skia interaction all happens here in this one function: https://github.com/aclysma/skulpin/blob/e20b8ced82d33538d9c54684a2b4e63eff6af450/skulpin-renderer/src/skia_renderpass.rs#L602
get_image_from_skia_texturegets a pointer to the image that the skia canvas will draw on- the first
add_image_barrier"borrows" it - everything between
cmd_begin_render_passandcmd_end_render_passjust draws a full-screen quad, textured with the image from skia add_image_barriergives it back to skia
If you already have a windowing framework and want to manage your own vulkan surface, adding skia to it is not too difficult. You will probably need to do this at some point anyways as I think you'll want direct control on when/how you present images.
lehmanju
Hey, I am a developer at Xournal++ and we are currently developing a library that includes Vulkan 2D rendering in Rust. The library should work on many platforms and allow easy development of new applications. A new application would only need to provide a graphical environment and an event loop, all events and a Vulkan surface would belong to our library. As each element is rendered by a corresponding plugin, we let each plugin render their element and arrange them afterwards on the viewport. If some element is selected or a plugin specific tool is active, we let the plugin render onto the complete viewport. This would allow playback of videos or other custom editing widgets to be used. Continuous rendering by plugins would overlay static rendering of the viewport. This looks very similar to #30 thus my request.
Unfortunately, I have no experience with Vulkan and have only read the tutorial to understand basic terminology. Would skulpin allow such a plugin based system and how would integration into Qt/GTK/etc. work? Could skulpin expose a raw Vulkan area for plugins to render videos on?