Compilation of `spirv-opt` to WASM

[wdanilo]

Hi! Is it possible to compile spirv-opt to WASM? I was able to run ./source/wasm/build.sh, but according to docs, "The resulting SpirvTools WebAssembly module only exports methods to assemble and disassemble SPIR-V modules". My use case is that we are generating shaders at runtime and we want to be sure they are properly optimized before feeding them to WebGL, so we want to replicate this workflow in WASM:

glslc --target-env=opengl -fshader-stage=... -o ${FILE}.spv ${FILE}
spirv-opt -Os -o ${FILE}.spv.opt ${FILE}.spv
spirv-cross --output ${FILE}.opt ${FILE}.spv.opt

[dneto0]

The only reason it was limited to assembly and disassembly is that nobody put the work in to do it. That said the optimizer is quite large, so you may want to make a separate WASM module for the optimizer.

[mwu-tow]

Hi @dneto0, thank you for the answer! Could you provide us some pointers as to how to proceed with separating the optimizer module and how to compile it? We don't have much experience with CMake, so any hints would be very valuable.

[farmaazon]

I can answer this question, as I'm doing a similar thing.

The diff I have:

diff --git a/source/wasm/build.sh b/source/wasm/build.sh
index f02ae525..d9f5fda1 100755
--- a/source/wasm/build.sh
+++ b/source/wasm/build.sh
@@ -34,7 +34,7 @@ build() {
         -DCMAKE_BUILD_TYPE=Release \
         $args \
         ../..
-    emmake make -j $(( $NUM_CORES )) SPIRV-Tools-static
+    emmake make -j $(( $NUM_CORES )) SPIRV-Tools-opt

     echo Building js interface
     emcc \
@@ -43,6 +43,7 @@ build() {
         -std=c++11 \
         ../../source/wasm/spirv-tools.cpp \
         source/libSPIRV-Tools.a \
+        source/opt/libSPIRV-Tools-opt.a \
         -o spirv-tools.js \
         -s MODULARIZE \
         -Oz
diff --git a/source/wasm/spirv-tools.cpp b/source/wasm/spirv-tools.cpp
index 33f2f05f..f9dffc9c 100644
--- a/source/wasm/spirv-tools.cpp
+++ b/source/wasm/spirv-tools.cpp
@@ -13,6 +13,7 @@
 // limitations under the License.

 #include "spirv-tools/libspirv.hpp"
+#include "spirv-tools/optimizer.hpp"

 #include <iostream>
 #include <string>
@@ -50,9 +51,22 @@ emscripten::val as(std::string const& source, uint32_t env, uint32_t options) {
     ptr));
 }

+emscripten::val opt(std::string const& input, uint32_t env, uint32_t options) {
+  spvtools::Optimizer optimizer(static_cast<spv_target_env>(env));
+  optimizer.SetMessageConsumer(print_msg_to_stderr);
+
+  std::vector<uint32_t> optimized;
+  const uint32_t* input_ptr = reinterpret_cast<const uint32_t*>(input.data());
+  if (!optimizer.Run(input_ptr, input.size(), &optimized)) optimized.clear();
+  const uint8_t* output_ptr = reinterpret_cast<const uint8_t*>(optimized.data());
+  return emscripten::val(emscripten::typed_memory_view(optimized.size() * 4,
+    output_ptr));
+}
+
 EMSCRIPTEN_BINDINGS(my_module) {
   function("dis", &dis);
   function("as", &as);
+  function("opt", &opt);

   constant("SPV_ENV_UNIVERSAL_1_0", static_cast<uint32_t>(SPV_ENV_UNIVERSAL_1_0));
   constant("SPV_ENV_VULKAN_1_0", static_cast<uint32_t>(SPV_ENV_VULKAN_1_0));

No cmake needed to be built, we must just build a different target (the emmake command) and link the built static library to the interface.

Additionally, in source/wasm/spirv-tools.cpp you need to bind the functions you want to use in js using embind. For example, I've created a simple function opt for optimizing existing binary. Be careful, though: it duplicates the problem the as method has with returning a pointer to freed memory (see #5036)

[dneto0]

Thanks. Yesterday I was looking at making a separate node package for the optimizer because it is so large. I'm most of the way there, and will take care of the use-after-free.

[dneto0]

Also, the optimizer has a more complicated interface, where you can schedule specific passes, or entire recipes at a time. So I was working my way through providing that API.

[dneto0]

I have a WIP PR that adds a separate WASM package for the spir-v optimizer. The compressed wasm is 562kb. Which is a lot.

And the PR doesn't work. Something is being messed up earlier in the flow I think. The simple SPIR-V module assembled in the test-opt.js file doesn't disassemble.

I'd welcome feedback if I've messed up WASM-bindings or other emscripten things. I'm a total newbie in it.

[wdanilo]

Also, the optimizer has a more complicated interface, where you can schedule specific passes, or entire recipes at a time. So I was working my way through providing that API.

Oh wow, thank you so much @dneto0 ❤️ I was able to compile the WASM module with what @farmaazon provided above and the final WASM opt file is relatively small - 2Mb? I'm wondering if it contains all the passes though. Just out of curiosity, do you have any approximate timeline for your module? I'm asking only to plan how / when I could integrate it to what we are building 😄