vsg-dev / vsgExamples

Example programs that test and illustrate how to use the VSG and optional add-on libraries
MIT License
145 stars 67 forks source link

vsgsreenshot doesn't work correctly on windows 10 #141

Closed rainergericke closed 2 years ago

rainergericke commented 2 years ago

Hi Robert,

vsgscreenshot models/teapot.vsgt

shows the famous Melitta teapot as 3d scene. With the 's' key a 2d model of the actual view in the file 'screenshot.vsgt' can be written. Opening this file with:

vsgviewer screenshot.vsgt

shows only the background color on Windows. For our project we need the image export as basic functionality for video generation. This information is available in the variable 'imageData' and can easily be exported via stb_image_write. It works perfectly on my Mac M1 MacOS 12.3. On Windows (AMD Ryzen 9 with NVIDIA GeForce RTX 3050) only shows the background color.

For further tests I modified the vsgscreenshot example, it exports flat image files now. Can you repeat this error?

Best TestProblem.zip

robertosfield commented 2 years ago

HI Rainer,

I don't have an explanation at this point, only suggestions on what to test next. Could you run the example with -d to enable the Vulkan debug laye to see if there are any errors.

It's curious that you are seeing the background colour on the Windows image, so some frame buffer image is being read, just not one that contains the final image, so perhaps there is a sync issue of some kind such that the screen capture code is running too early in the frame.

W.r.t the stbi_write_png, I think it would be useful to add this to vsgXchange.

The code you've added to do the stbi_write_png creates a local char array then copies the data across. I don't think this is required as you should be able to get the data directly from vsg::Data::dataPointer(), you'll just need to cast the void to char.

Also as a general note on vsg::Data usage, the begin()/end() iterator support enables C++17's for() shorthand, so the original code:

   size_t k = 0;
    for(size_t i=0; i<imageData->size(); i++) {
        vsg::ubvec4 pix = imageData->at(i);
        pixels[k++] = pix[0];
        pixels[k++] = pix[1];
        pixels[k++] = pix[2];
        pixels[k++] = pix[3];
    }

Can be replaced by:

    for(auto& pix : *imageData) {
        pixels[k++] = pix[0];
        pixels[k++] = pix[1];
        pixels[k++] = pix[2];
        pixels[k++] = pix[3];
    }

Cheers, Robert.

Message ID: @.***>

rainergericke commented 2 years ago

Hi Robert,

thank you for the hints, I will try it.

Appened is the message from the windows version with '-d' flag. win_message.txt

Best Rainer

robertosfield commented 2 years ago

Hi Rainer,

On Wed, 16 Mar 2022 at 09:02, Rainer Gericke @.***> wrote:

Appened is the message from the windows version with '-d' flag.

win_message.txt https://github.com/vsg-dev/vsgExamples/files/8260414/win_message.txtMessage ID: @.***>

Ooo, that's interesting, your run is using the vkCmdBlitImage code path, I don't recall previously this path taken in my own testing so it could well be that there are errors in there I haven't seen. It could be that Mac is working for you because it's not selected the BlitImage path.

Could you hack the code so it doesn't use BliImage - simply setting supportsBlit to false would be a place to start.

Robert.

rainergericke commented 2 years ago

Hi Robert,

here come the Mac results. I user the '-d' flag here also. Switching off the blitter code, the program still works but changes the background color.

The CAMetalLayer related message always occurs, but does hopefully no harm.

Best

Rainer mac_blitter_message.txt mac_noblitter_message.txt screenimage_mac_blitter screenimage_mac_noblitter screenshot_mac_blitter.vsgt.zip screenshot_mac_noblitter.vsgt.zip

rainergericke commented 2 years ago

...the color seems to be switched, when no blitter is active.

rainergericke commented 2 years ago

Hi Robert,

I found a screenshot example, which works on Mac and on Windows. It is part of Sascha Willems vulkan examples. [(https://github.com/SaschaWillems/Vulkan.git)] I looked at the save_screenshot() code and compared it to the code in the vsgscreenshot example. I see some differences regarding parameters like 'VK_QUEUE_FAMILY_IGNORED', but I am not an expert to say what is wrong. Maybe this example can help you?

robertosfield commented 2 years ago

Hi Rainer,

On Sun, 10 Apr 2022 at 09:49, Rainer Gericke @.***> wrote:

I found a screenshot example, which works on Mac and on Windows. It is part of Sascha Willems vulkan examples.

[(https://github.com/SaschaWillems/Vulkan.git)] I looked at the save_screenshot() code and compared it to the code in the vsgscreenshot example. I see some differences regarding parameters like 'VK_QUEUE_FAMILY_IGNORED', but I am not an expert to say what is wrong. Maybe this example can help you?

I am already rather stretched thin trying to complete core features and tackle other outstanding issues so not in a good position to dive into macOS issues. I don't have a macOS system or any expertise on it. The best I can do is suggest things to try - for instancing running the Sasha Willems example with vulkan api/debug layer and the vsg one with api/debug layer to compare the settings then tweak the VSG example so if you can can close the gap and get it working.

Message ID: @.***>

rainergericke commented 2 years ago

Hi Robert,

this is a Windows issue, but I know, your answer will be the same for the same reasons. I will try what is possible. I am excited to see the new core features yet!

robertosfield commented 2 years ago

@rainergericke Did you uncover what is happening with the screenshot issue under Windows?

rainergericke commented 2 years ago

No, I pushed it behind several times. It would be good to have this problem resolved yet. Do you have time to help?

rainergericke commented 2 years ago

I have a prelaminary solution for Windows:

  1. Ignore Blitting support
  2. Consider color swizzle and non contiguity

As attachment you find my screenshot function for Windows. The output code was taken from Sascha Willems screenshot example, only meant as a demo, but working exportScreenshotNoBlit.cpp.zip .

robertosfield commented 2 years ago

Thanks for the example code. Apart from missing the BlitImage codepath does the Vulkan related code differ?

Is the color swizzle just required because the PPM format you are writing to is RGB?

Message ID: @.***>

rainergericke commented 2 years ago

I didn't change the Vulkan code, it should be identical.

Using your code on the Mac with blitting switched off, you can see the image but in wrong colors, so it is necessary to retrieve information about color swizzle. Using the code on Windows seems to have right colors, but the image is scrambled because the data is not contiguous. So it is better to consider both.

PPM expects RGB, also stb want RGBA.

robertosfield commented 2 years ago

On Mon, 4 Jul 2022 at 08:25, Rainer Gericke @.***> wrote:

I didn't change the Vulkan code, it should be identical.

Using your code on the Mac with blitting switched off, you can see the image but in wrong colors, so it is necessary to retrieve information about color swizzle. Using the code on Windows seems to have right colors, but the image is scrambled because the data is not contiguous. So it is better to consider both.

Is it the retrieved frame buffer than is non-contiguous?

Does your example non non-contiguous data?

PPM expects RGB, also stb want RGBA.Message ID:

@.***>

Swizzle will need to be done in the writer so I see that as a different topic.

Currently the vsgXchange/stbi integration doesn't have a write code path which would be useful.

robertosfield commented 2 years ago

To make it easier to write images I've created a stbi_write branch of vsgXchange, and have add stb_image_write.h as a first step to adding support for writing image formats:

[https://github.com/vsg-dev/vsgXchange/tree/stbi_write](https://github.com/vsg-dev/vsgXchange/tree/stbi_write)

I will now figure out how to use the stbi_image_write API, I don't expect it'll be too hard.

robertosfield commented 2 years ago

I have now checked in support for writing RGB, RGBA image formats to the stbi_write https://github.com/vsg-dev/vsgXchange/tree/stbi_write branch of the vsgXchange. stbi doesn't look to have any support for specifying BGR/BGRA formats or any swizzle support so I'll implement that next.

robertosfield commented 2 years ago

I have added swizzling from BGR and BGRA to RGB and RGBA respectively:

Added support for swizzling image format to handle BGR and BGRA formats

I will do a code review and then if it looks OK merge with vsgXchange master.

robertosfield commented 2 years ago

The png, jpeg, bmp and tga write support for RGB, RGBA, BGR and BGRA, the later with a locally implemented swizzle is now merged into vsgXchange master.

rainergericke commented 2 years ago

Great! It is easy to use. It works well on the Mac. On Windows there is still a bug with blit support enabled, which leads to an one-colored graphic file (all formats, vsg* included). Disabling the blitter makes it run well. I will use this function in my software.

robertosfield commented 2 years ago

Could you try running vsgscreeenshot with debug layer enabled i.e.

vsgscreenshot models/lz.vsgt --cf screenshot.png -d

On Windows and macOS? I'd like to get to bottom of the issue with the blitting.

robertosfield commented 2 years ago

I have just run vsgscreenshot with debugging on and see errors, so I have something to look into at my end.

$ vsgscreenshot models/lz.vsgt -d depthStencilResolve_properites.supportedDepthResolveModes = 15 depthStencilResolve_properites.supportedStencilResolveModes = 13 depthStencilResolve_properites.independentResolveNone = 1 depthStencilResolve_properites.independentResolve = 1 info: supportsBlit = 1 VUID-vkCmdBlitImage-srcImage-00247(ERROR / SPEC): msgNum: 1607722825 - Validation Error: [ VUID-vkCmdBlitImage-srcImage-00247 ] Object 0: handle = 0x556b8b9b4130, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x5fd3e749 | vkCmdBlitImage: region [0], source image of type VK_IMAGE_TYPE_1D or VK_IMAGE_TYPE_2D with srcOffset[].z values of (0, 0). These must be (0, 1). The Vulkan spec states: If srcImage is of type VK_IMAGE_TYPE_1D or VK_IMAGE_TYPE_2D, then for each element of pRegions, srcOffsets[0].z must be 0 and srcOffsets[1].z must be 1 (https://vulkan.lunarg.com/doc/view/1.3.216.0/linux/1.3-extensions/vkspec.html#VUID-vkCmdBlitImage-srcImage-00247) Objects: 1 [0] 0x556b8b9b4130, type: 6, name: NULL VUID-vkCmdBlitImage-dstImage-00252(ERROR / SPEC): msgNum: -657803409 - Validation Error: [ VUID-vkCmdBlitImage-dstImage-00252 ] Object 0: handle = 0x556b8b9b4130, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0xd8cab76f | vkCmdBlitImage: region [0], dest image of type VK_IMAGE_TYPE_1D or VK_IMAGE_TYPE_2D with dstOffset[].z values of (0, 0). These must be (0, 1). The Vulkan spec states: If dstImage is of type VK_IMAGE_TYPE_1D or VK_IMAGE_TYPE_2D, then for each element of pRegions, dstOffsets[0].z must be 0 and dstOffsets[1].z must be 1 (https://vulkan.lunarg.com/doc/view/1.3.216.0/linux/1.3-extensions/vkspec.html#VUID-vkCmdBlitImage-dstImage-00252) Objects: 1 [0] 0x556b8b9b4130, type: 6, name: NULL Written color buffer to screenshot.vsgt num_unset_depth = 1276732 num_set_depth = 33988 Written depth buffer to depth.vsgt

rainergericke commented 2 years ago

This is the Mac answer (Windows will follow soon): rainerge@RainersacStudio VSG2 % vsgscreenshot models/lz.vsgt -d depthStencilResolve_properites.supportedDepthResolveModes = 13 depthStencilResolve_properites.supportedStencilResolveModes = 1 depthStencilResolve_properites.independentResolveNone = 1 depthStencilResolve_properites.independentResolve = 1 VUID-vkCreateEvent-events-04468(ERROR / SPEC): msgNum: 2064270843 - Validation Error: [ VUID-vkCreateEvent-events-04468 ] Object 0: handle = 0x143045e18, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x7b0a45fb | vkCreateEvent: events are not supported via VK_KHR_portability_subset The Vulkan spec states: If the VK_KHR_portability_subset extension is enabled, and VkPhysicalDevicePortabilitySubsetFeaturesKHR::events is VK_FALSE, then the implementation does not support events, and vkCreateEvent must not be used (https://vulkan.lunarg.com/doc/view/1.3.216.0/mac/1.3-extensions/vkspec.html#VUID-vkCreateEvent-events-04468) Objects: 1 [0] 0x143045e18, type: 3, name: NULL

robertosfield commented 2 years ago

On Mon, 4 Jul 2022 at 16:15, Rainer Gericke @.***> wrote:

This is the Mac answer (Windows will follow soon): @.*** VSG2 % vsgscreenshot models/lz.vsgt -d depthStencilResolve_properites.supportedDepthResolveModes = 13 depthStencilResolve_properites.supportedStencilResolveModes = 1 depthStencilResolve_properites.independentResolveNone = 1 depthStencilResolve_properites.independentResolve = 1 VUID-vkCreateEvent-events-04468(ERROR / SPEC): msgNum: 2064270843 - Validation Error: [ VUID-vkCreateEvent-events-04468 ] Object 0: handle = 0x143045e18, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x7b0a45fb | vkCreateEvent: events are not supported via VK_KHR_portability_subset The Vulkan spec states: If the VK_KHR_portability_subset extension is enabled, and VkPhysicalDevicePortabilitySubsetFeaturesKHR::events is VK_FALSE, then the implementation does not support events, and vkCreateEvent must not be used ( https://vulkan.lunarg.com/doc/view/1.3.216.0/mac/1.3-extensions/vkspec.html#VUID-vkCreateEvent-events-04468 ) Objects: 1 [0] 0x143045e18, type: 3, name: NULL

Um.... that's an curve ball if vsg::Event isn't supported. Which version of Vulkan are you working with on macOS? I will have a think about what might be done.

--

On a positive note, I've fixed the Vulkan debug error relating to vsg::BlitImage settings and checked it in: 06bb1f6d27c59d78ee1d240acfbe6b156c6dec1c

Could you check out vsgExamples master and then redo the windows test?

Message ID: @.***>

rainergericke commented 2 years ago

It's Vulkan 1.3.216.0.

Windows doesn't tell us to much: windows_scrshot.txt

robertosfield commented 2 years ago

On Mon, 4 Jul 2022 at 16:28, Rainer Gericke @.***> wrote:

It's Vulkan 1.3.216.0.

So it's using the new enabling of portability_subset. I will try adding an option to switch off the use of the vsg::Event/SetEvent.

Windows doesn't tell us to much: windows_scrshot.txt https://github.com/vsg-dev/vsgExamples/files/9040681/windows_scrshot.txt

If made after I made the fix?

How does the screenshot look under Windows and macOS now?

Message ID: @.***>

robertosfield commented 2 years ago

I have added a --use-vkEvent command line option to enable the use a vsg::Event/vkEvent within the screenshot code, with it defaulting to off: 2cf18197ca0b2240f236efea3d10eb2cc41dc43b

Could you retest under macOS and Windows and see how things are running. Without the vkEvent I'm not seeing any issues on my Linux AMD 5700G system, it's all working as before.

Message ID: @.***>

rainergericke commented 2 years ago

Looks good! screenshot_mac.vsgt.zip screenshot_win.vsgt.zip

robertosfield commented 2 years ago

On Mon, 4 Jul 2022 at 16:51, Rainer Gericke @.***> wrote:

Looks good! screenshot_mac.vsgt.zip https://github.com/vsg-dev/vsgExamples/files/9040818/screenshot_mac.vsgt.zip screenshot_win.vsgt.zip https://github.com/vsg-dev/vsgExamples/files/9040819/screenshot_win.vsgt.zip

Excellent, thanks for the testing. Does this mean that all the issues are resolved and we can now close to Issue?

Message ID: @.***>

rainergericke commented 2 years ago

Yes, we can close it. Thanks!