Android testing slowdown after shader val cache commit

[aabdolrashidi]

There has been a slowdown in testing ANGLE on Pixel 6 after the following commit: https://github.com/KhronosGroup/Vulkan-ValidationLayers/commit/efcb50f63a4cd7b921fe834a4a58ac3af42fa334 A change in Android test runtime can also be seen by disabling shader_validation_caching in core_validation.cpp (disabled[shader_validation_caching] = true). More info: Here

CC: @lunarpapillo @Tony-LunarG

[TonyBarbour]

Is there a better strategy for determining where to put the cache on Android?

[cnorthrop]

I'm guessing that things are behaving differently on Android because you added $TMPDIR, which is actually defined on Android. I doubt /data/local/tmp is actually writeable though, so there's probably no caching going on.

A better location is to write files to the application's sdcard directory. For example /sdcard/Android/data/package_name. Here's how we do it in ANGLE for writing out capture files. See GetDefaultOutDirectory in the link below. https://chromium.googlesource.com/angle/angle/+/HEAD/src/libANGLE/capture/FrameCapture.cpp#130

It would be good to see data that shows caching is helpful though. We could just turn it off.

[cnorthrop]

I misread the diff, $TMPDIR has always been checked. Not sure what is causing the slowdown.

In any case, I did some experiments running one of our end2end tests that does mostly shader work, recording the time taken. These are from a Pixel 6 Pro:

Test:
out/AndroidDebug/angle_end2end_tests --gtest_filter="*GLSLTest*Fragment*ES3_Vulkan" --verbose

Default VVL (top of tree):
9964
10200
10324

Remove TMPDIR check:
9615
9671
9397

Revert the CL:
8992
8766
8927

Disable shader caching:
6941
6784
7272

I went ahead and modified the caching location so files could be created and the layers would get a hit. First I created the following location:

adb shell mkdir /sdcard/vulkan_validation_cache
adb shell chmod 777 /sdcard/vulkan_validation_cache

and hard coded it in the layer:

tmp_path = "/sdcard/vulkan_validation_cache";

That resulted in the following file being created:

$ adb shell ls -la /sdcard/vulkan_validation_cache/
total 4
-rw-rw---- 1 u0_a251 media_rw 76 2022-02-24 23:09 shader_validation_cache-10258.bin

Running it with the test, I see:

Fix shader caching:
9204
9130
9305

So I don't think caching is worth the trouble on Android.

Is there a recommended way to disable it without having to use environment variables? I'm hard coding like so just before the check for it:

disabled[shader_validation_caching] = true;

[ncesario-lunarg]

@cnorthrop is using the vk_layer_settings.txt config file a viable option for setting VK_VALIDATION_FEATURE_DISABLE_SHADER_VALIDATION_CACHE_EXT (although, not sure if shader caching is an exception here @Tony-LunarG?)?

[TonyBarbour]

That should work, and we can ifdef the default to false if desired

[cnorthrop]

@ncesario-lunarg I don't believe vk_layer_settings.txt works on Android, but @mark-lunarg is looking in to it.

@Tony-LunarG I don't think you should force it off for Android. There may be devices running Android that benefit from it. I do think it is worth getting the cache to work using a spot the app can access. I can help with that.

For ANGLE, we can disable caching programmatically through a pNext chain.

[TonyBarbour]

Ok, I'll abort my disable PR :-) For the record, on Windows we saw setup times drop significantly for the worst case game traces: dota 531-541 - 7 seconds to 2 seconds dota 5155-5165 - 13 seconds to 4 seconds doom 5793-5802 - 7 seconds to 4 seconds

The change in question only added XDG_CACHE_HOME and HOME/.cache to the front of the list of locations to try. This was for better security on multi-user systems. Let me know if there is a better location list for Android and I can use that instead.

[TonyBarbour]

@cnorthrop any suggestions for a remedy?

[TonyBarbour]

Closing due to inactivity. Feel free to re-open if this is still an issue