Automatically checking best practices

[HansKristian-Work]

Vulkan being a more explicit API means more room to make poor decisions for performance which can be hard to catch during development. It would be interesting if we could have a performance checking layer which can find the most common issues for different IHVs in a common layer. There is one vendor specific layer out in the wild: https://github.com/ARM-software/perfdoc

[KarenGhavam-lunarG]

LunarG has created a VLLF (Vulkan Lightweight Layer Framework) and is using the VLLF to create an "Assistant" layer to check for poor usage. We have been collecting suggestions of things to track in this LVL issue. @mark-lunarg is working on this and can elaborate more about the features and it's current status.

performance checking layer which can find the most common issues for different IHVs in a common layer.

I believe the layer is currently targeted to catch bad usage applicable for all IHVs. Mark can elaborate.

[mark-lunarg]

@HansKristian-ARM (and everyone else), by all means, please review the tracking issue that @KarenGhavam-lunarG pointed to and add your observations or suggestions. Not only will the assistant layer provide coverage for many of the Vulkan performance/practice pitfalls, use of the layer framework should allow users to add customizations quickly and easily.

[HansKristian-Work]

Based on discussions yesterday, it is clear that if we want to progress on this, we will need to convince IHV that it is worthwhile to do this. One start would be to better promote the current work in this area I think.

[KarenGhavam-lunarG]

In a previous comment I mentioned the VLLF (now called VLF) and wanted to provide an update on this tool that I believe is part of the solution to this issue.

The Vulkan Layer Framework (VLF) allows developers to quickly and easily create layers to aid in application development. The frame makes it possible to create a throwaway layer for just about any purpose. As well, it highly simplifies the creation of full-featured layers by abstracting the extensive and complicated standard layer setup and initialization for the user.

Some simple VLF Use Cases:

• ISV: We regularly receive requests from application developers for simple additions to validation layers that would enable them to facilitate their workflow. The VLF can be used by ISVs to add their own custom additions enabling tighter integration with proprietary debugging tools.
• IHV: The VLF enables the ability to create quick, one-off interceptors to investigate application or driver issues. A built-in breakpoint facility allows them to stop execution in the application before getting to the driver, and frequency or timing profiling data is easily collected without modification of the driver or the application.

There is some suggestion that a tool simplifying layer creation could help motivate IHVs to create hardware specific "best practice" monitoring layers.

The VLF is found in LunarG/VulkanTools in the Layer Factory directory, and will be available in an upcoming Vulkan SDK.

In addition, we have begun implementing "best practice" checks as collected in github issue 1612 . These best practice checks are being put into the "Assistant Layer" that is also found in the LunarG/VulkanTools repo in the assistant_layer directory and will be delivered in a future Vulkan SDK.

[natduca]

I guess I'll jump in and say I personally think the core issue here is developers swamped with advice. There exists a world where we make authoring layers easy where each gpu vendor has a layer and poor game devs now have to know where to find each of these layers and bring it together.

In contrast, Microsoft is just "making it work" in pix. https://blogs.msdn.microsoft.com/directx/2017/03/01/gpu-plugins-improved-sdk-layers-and-hang-debugging-bringing-directx-12-tools-to-the-next-level/

I think its worth us talking about the end game here, then doing what we can to make our individual technical choices that we make along the way take us to the end game.

[HansKristian-Work]

From what I can tell, that's exposing hardware counters through vendor-specific plugins, which is a separate issue to me.

If the model of creating layers to solve individual problems is considered flawed, then that places the entire layering system under question. I think it's more of a packaging problem than anything.

For an integrated experience where a single tool can handle everything, we'd still need to see IHVs committing to a single debugger and some kind of plugin interface.

[nsubtil]

While discussing #11 we reached the conclusion that many of the same factors surrounding that issue also apply here. We're trying to figure out what the right way to expose this kind of information is, and where this functionality should be (see https://github.com/KhronosGroup/Vulkan-Ecosystem/issues/11#issuecomment-371573864 for more details).

[KarenGhavam-lunarG]

LunarG had some internal discussions about this issue as well as issue #11. Our recommendation is the following:

1. The Assistant layer is to do best practice checks that are generally applicable to all Vulkan applications.
   • You could extend the Assistant Layer to do these portability checks (or create a portability layer to do these checks). I added the example checks that @HansKristian-ARM has provided in this issue to the Assistant Layer tracking issue #1612.
   • LunarG has resources applied in this area and intends to continue to enhance the Assistant Layer.
   • Tracking of progress will be kept up to date in the Assistant Layer tracking issue #1612
2. Best practice/performance checks that are device specific should be done by IHV specific layers.
   • It would be great if the IHVs could use the VLF (Vulkan Layer Factory) to create such layers.
   • ARM has a device specific layer it may help motivate other IHVs to contribute layers if the current ARM layer is ported to use the VLF. The porting activity to the VLF could be used to demonstrate the value of the VLF tool to make layer writing easy. LunarG may have upcoming cycles to port this layer to the VLF and may consider doing so.
   • LunarG is certainly willing to provide consultation and assistance to an IHV who would like to use the VLF to create an IHV specific best practices layer.

So perhaps we should close out issue #7 and issue #11 since they are being tracked in theAssistant Layer tracking issue #1612? And then we create a new issue focused on IHV/vendor specific best practices/performance checks. By having this new focused issue, perhaps the "Ecosystem forum team" can reach out to the IHVs to help influence them to create such layers?

[marty-johnson59]

This repository is being archived as it has been replaced with the vulkan.org website (https://www.vulkan.org) and is no longer being maintained (i.e., issues posted here are no longer being addressed ). After reviewing issues posted here, most (if not all) have been resolved or have already been re-opened in Vulkan-Docs (https://github.com/KhronosGroup/Vulkan-Docs) or other repositories for further consideration. Therefore, all issues in this repository will be closed. If you believe your issue has not yet been resolved, please re-open in Vulkan-Docs. Thanks!