Code traceability strategies

[Ibnmardanis24]

This issue would like to raise 3 main points derived from my brief experience toying around with the compiler:

• I feel that optimization features should be off by default. Choosing to apply optimizations should be an active selection from the end-user, rather than the opposite.
• Even after setting off every opt option I could find, the -dump-pass-name flag reveals that around 200 passes are still applicable. I was wondering whether any of these could be further removed when in a no optimization scenario. My goal is to find the "most reduced subset" of compiler passes capable of bringing a particular shader from SPIR-V to a valid and representative AMD GCN kernel. In order to be more specific, I am particularly concerned with compute shaders and their offline pipelines.
• Finally, my end interest is on developing tracing and debugging strategies using this compiler as sandbox. Any and all tools already present in this project, even unfinished ones, are interesting for me to learn and understand.

I am sorry if this was not the right forum for such requests, but I could not find any alternatives...

[Flakebi]

Vulkan expects the default to be optimized and as llpc is the compiler frontend for Vulkan, it’s following that :)

The amdllpc offline compiler supports command line options to disable optimizations, like -llpc-opt=quick (equivalent of -O1, there’s also -llpc-opt=none, though beware that this is untested and sometimes creates buggy code). And, from the Vulkan API, one can set VK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BIT to disable most optimizations.

[Ibnmardanis24]

Interesting points.

Following on your comment,

Vulkan expects the default to be optimized and as llpc is the compiler frontend for Vulkan, it’s following that :)

In what sense does Vulkan require optimized shaders as input? Better even, where in the standards is this specified?

Coming from an entirely different background this seems really weird to me, any reference to the docs is appreciated!

The amdllpc offline compiler supports command line options to disable optimizations, like -llpc-opt=quick (equivalent of -O1, there’s also -llpc-opt=none, though beware that this is untested and sometimes creates buggy code).

These are the kind of options I was mentioning on my first post (I believe there is also a -opt=none that is passed to LLVM). I suppose you left the none selection untested because of this Vulkan interaction. However, wouldn't it be easier to test the offline compiler without optimizations than having a big pile of transformations for testing? In any case, this maybe interesting anyway, what would it take to create confidence around the compiler's non-optimizing version?

[Flakebi]

In what sense does Vulkan require optimized shaders as input? Better even, where in the standards is this specified?

Vulkan doesn’t really require optimizations, because as a vendor-agnostic API, it can’t really define what “optimizations” are. So, it’s up to the driver to decide what it wants to do. The spec explains the disable optimization flag though (link):

VK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BIT specifies that the created pipeline will not be optimized. Using this flag may reduce the time taken to create the pipeline.

However, wouldn't it be easier to test the offline compiler without optimizations than having a big pile of transformations for testing?

The offline compiler is a tool to test the compiler that is part of the Vulkan driver, so I’d argue we want to test whatever is used most in the driver. For a graphics driver it’s usually beneficial to optimize shaders because they will be compiled once and executed thousands to billions of times (thinking of a pixel shader that runs once over a 1920×1080 image at 60 fps, which makes 124 million invocations per second, 4k@144 Hz would be 1.2 billion invocations per second).

For games, disabling optimizations can be used to reduce compile time and stuttering, while still compiling an optimized version in the background: https://www.phoronix.com/scan.php?page=news_item&px=RADV-CREATE_DISABLE_OPT_BIT

[Ibnmardanis24]

I completely agree with you on the importance of optimizing graphic shaders. In fact, you are also right, these are going to be the vast majority of the use cases that this driver is ever going to face.

Allow me, nevertheless, to diverge from this line and focus on compute shaders. My interest is not being fast, but rather, being always correct. This is why I keep asking about removing unnecessary transformations. I believe this exercise could also provide an edge to the optimizing case, sort of a consolidated background. On that sense I end up here again,

what would it take to create confidence around the compiler's non-optimizing version?

Let's entertain the idea of testing the non-optimizing case for a moment.

How are you testing the compiler under optimizing settings? What kind of metrics should I look for? I presume that if you have always lived with optimizations, you won't have any traceability tools to aid you, so I wonder what those tests look like.

If you are curious about why in the world would I interest myself on this, we could have a brief chat about it anytime. :D

[Flakebi]

My interest is not being fast, but rather, being always correct.

I agree with that, -O0 not working is a bug. To add some context, there were llvm (amdgpu backend) bugs with -O0. Some of them were fixed but I think nobody did enough testing to be confident that there are no more of these bugs. Which smoothly leads to your next question

How are you testing the compiler under optimizing settings?

For Vulkan, that is running the Conformance Test Suite (and some additional testing internally at AMD). So, a passing CTS when forcing -O0 would be nice. It just needs someone doing the work ;)