Issues with WASM Support

SamKouteili commented 10 months ago

I'm attempting to generate a very simple MathUtils class with one function: add_fff64, from c++ to JavaScript.

#include "MathUtils.hpp"

class MathUtils_ : public MathUtils {
public:
    double add_fff64(double a, double b) override {
        return a + b;
    }
};

Using Snapchat/Djinni's wasm interface, I've generated relevant binding files, and have also generated the cpp header:

// ***wasm/MathUtils.hpp***
// AUTOGENERATED FILE - DO NOT MODIFY!
// This file was generated by Djinni from MathUtils.djinni

#pragma once

#include "src/MathUtils.hpp"
#include "snap.djinni/support-lib/wasm/djinni_wasm.hpp"

namespace djinni_generated {

struct MathUtils : ::djinni::JsInterface<::MathUtils, MathUtils> {
    using CppType = std::shared_ptr<::MathUtils>;
    using CppOptType = std::shared_ptr<::MathUtils>;
    using JsType = em::val;
    using Boxed = MathUtils;

    static CppType toCpp(JsType j) { return _fromJs(j); }
    static JsType fromCppOpt(const CppOptType& c) { return {_toJs(c)}; }
    static JsType fromCpp(const CppType& c) {
        ::djinni::checkForNull(c.get(), "MathUtils::fromCpp");
        return fromCppOpt(c);
    }

    static em::val cppProxyMethods();

    static double add_fff64(const CppType& self, double w_a,double w_b);

};

} // namespace djinni_generated

// ***wasm/MathUtils.cpp***
// AUTOGENERATED FILE - DO NOT MODIFY!
// This file was generated by Djinni from MathUtils.djinni

#include "wasm/MathUtils.hpp"  // my header

namespace djinni_generated {

em::val MathUtils::cppProxyMethods() {
    static const em::val methods = em::val::array(std::vector<std::string> {
        "addFff64",
    });
    return methods;
}

double MathUtils::add_fff64(const CppType& self, double w_a,double w_b) {
    try {
        auto r = self->add_fff64(::djinni::F64::toCpp(w_a),
                  ::djinni::F64::toCpp(w_b));
        return ::djinni::F64::fromCpp(r);
    }
    catch(const std::exception& e) {
        return ::djinni::ExceptionHandlingTraits<::djinni::F64>::handleNativeException(e);
    }
}

EMSCRIPTEN_BINDINGS(_MathUtils) {
    em::class_<::MathUtils>("MathUtils")
        .smart_ptr<std::shared_ptr<::MathUtils>>("MathUtils")
        .function("nativeDestroy", &MathUtils::nativeDestroy)
        .function("addFff64", MathUtils::add_fff64)
        ;
}

} // namespace djinni_generated

While this has indeed compiled into a JavaScript output, there is no semblance of the MathUtils class or any add_fff64 function in MathUtils.js. Indeed, when I attempt to call the add_fff64 function in a generic native html file, I get an error. I use the following script: emcc wasm/MathUtils.cpp src/MathUtils.cpp /Users/sam/snap.djinni/support-lib/wasm/djinni_wasm.cpp -o MathUtils.js --bind

This is particularly strange to me because when debugging the generated WebAssembly file with tools such as wasm-objdump and wasm2wat, MathUtils class and addFff64 both appear.

...
 - 0010490: 7562 6c65 3e00 6164 6446 6666 3634 0048  uble>.addFff64.H
 ...

Is there any way of confirming what may be the root cause of the issue?

LiFengSC commented 10 months ago

How do you call the function from JS in your html file? Did you load the module and access the function through the module object? Here's an example of calling wasm module from JS https://github.com/Snapchat/djinni/blob/main/test-suite/handwritten-src/js/test.html#L40

SamKouteili commented 10 months ago

I attempt to call it in a simple HTML file index.html in the same directory as the js and wasm output:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>MyClass Test</title>
</head>
<body>
    <script src="MathUtils.js"></script>
    <script>
        Module.onRuntimeInitialized = function() {
            var instance = new Module.MathUtils();
            console.log("successfully instantiated");
            console.log(Module.MathUtils.addFff64(10, 8));
            instance.delete();
        };
    </script>
</body>
</html>

I get the following error on my browser:

MathUtils.js:2182 Uncaught (in promise) BindingError: MathUtils has no accessible constructor
    at ClassHandle.<anonymous> (MathUtils.js:2182:21)
    at new MathUtils (MathUtils.js:1714:23)
    at Module.onRuntimeInitialized (index.html:11:28)
    at doRun (MathUtils.js:3963:71)
    at run (MathUtils.js:3980:5)
    at runCaller (MathUtils.js:3924:19)
    at removeRunDependency (MathUtils.js:623:7)
    at receiveInstance (MathUtils.js:842:5)
    at receiveInstantiationResult (MathUtils.js:860:5)

Should I be adding a constructor to my djinni interface? That also hasn't seemed to work. I've tried changing the script in the html file to:

        Module().then(module => {
          console.log(module);
        //   runTests(module, allTests);
      })

But get the error "module is not a function".

As an aside, Ive also tried making it so that add_fff64 is a static function, and ammending MathUtils.cpp to the following:

// src/MathUtils.cpp
#include "MathUtils.hpp"

double MathUtils::add_fff64(double a, double b) {
        return a + b;
}

However, I get a compilation error as an add_fff64 function is defined in both src/MathUtils.cpp and wasm/MathUtils.cpp. How are we expected to define static functions when exporting to js?

LiFengSC commented 10 months ago

I think you probably missed the "-s MODULARIZE=1" flag in your emcc command line. See the flags we use for compiling wasm code here https://github.com/Snapchat/djinni/blob/master/test-suite/BUILD#L148

SamKouteili commented 10 months ago

I just added the flag, and indeed now no longer face any error message, but also have nothing printed to the console, so this seems a little difficult to debug. When I instead changed my script to:

        Module().then(module => {
          console.log(module);
      })

There is no semblance of an addFff64 method. Indeed, (typeof module.addFff64 === 'function') returns a false. I've added my repository to the following repo: https://github.com/SamKouteili/djinni2emcc for reproduction. Hoping to write a blog post detailing how to go about doing this so future devs do not face the same issues :)

LiFengSC commented 10 months ago

Do you have module.MathUtils.addFff64 ?

See the generated binding here:

EMSCRIPTEN_BINDINGS(MathUtils) {
  class_<MathUtils>("MathUtils")
    .constructor<int>()
    .class_function("addFff64", &MathUtils::add_fff64)
    ;
}

Your function is inside MathUtils

SamKouteili commented 10 months ago

Unfortunately, module.MathUtils.addFff64 is also not defined. When I run the following:

                console.log(typeof module);
                console.log(typeof module.MathUtils);
                console.log(module.MathUtils.addFff64);

I get

object
function
[undefined]

I assumed this may be because addFff64 is not a static method (this is the final intended behaviour, but I still am not sure how to get around the compiler issue of having add_fff64 defined in both src/MathUtils.cpp and the generated wasm/MathUtils.cpp). As such, I tried instantiating an instance of MathUtils:

                console.log(typeof module);
                console.log(typeof module.MathUtils);
                console.log(module.MathUtils.addFff64);
                const mathUtilsInstance = new module.MathUtils();
                console.log(mathUtilsInstance);
                console.log(mathUtilsInstance.addFff64(1, 2));

But received an error that there is no valid constructor for the MathUtils module.

LiFengSC commented 10 months ago

Your function is not a static one:

MathUtils = interface +c {
    add_fff64(a: f64, b: f64): f64;  # this is an instance method
}

If you want to directly call the function without an object instance, you need to define the method as static:

MathUtils = interface +c {
    static add_fff64(a: f64, b: f64): f64;
}

and implement the static method in C++:

double MathUtils::add_fff64(double a, double b) {
        return a + b;
}

LiFengSC commented 10 months ago

If you want to use object instances, you would do something like this:

MathUtils = interface +c {
    add_fff64(a: f64, b: f64): f64;
    static create_math_utils(): MathUtils;
}

Then implement the static factory method in C++:

std::shared_ptr<MathUtils> MathUtils::create_math_utils() {
        return std::make_shared<MathUtils>();
}

SamKouteili commented 10 months ago

Fantastic, these were exactly the issues I was facing. I was able to generate both a static version where add_fff64 is a static function that can be called without instantiation, as well as a version where MathUtils needed to be initialized. Thank you truly for all the help and patience throughout! Will be publishing findings in some capacity so that other developers learn from the mishaps I've faced.

Snapchat / djinni

Issues with WASM Support #150