Accessing all methods / properties / constructors externally

[Mudloop]

Question

Hi,

I’m experimenting with a system that automatically creates Proxies for manipulating assemblyscript memory. It generates a constructable proxy per class, which can then be instantiated and that creates an object in wasm, keeps track of the pointer (with pinning / releasing) and it has getters / setters for the data (which return new proxies for objects, and caches them).

To achieve this, I made a build step using a transform which adds accessor functions for everything I need access to, but there might be a better way. I tried enabling “export table” but the table always only has 2 entries, the first of which is null, and I can’t find much information on this.

The second part of this system is generating metadata (ie reflection data) about field order and types, class ids, parameters etc, which I prefer during the transform phase, and I just generate a bunch of code and start the build over with the patched files.

It’s al a bit fragile doing both these things manually since there’s a bunch of features that I haven’t used before or that I’m not even aware of which likely could break things.

I believe it would be extremely useful if assemblyscript has an option to auto generate a json file with info on all classes, and - if that’s not already possible - ways to externally call any function.

Maybe the easiest way for the external calls would be adjusting the .wat file so everything gets exported, and using that?

[JairusSW]

Hey, @Mudloop Here's a few resources that might help you https://github.com/JairusSW/as-json/tree/master/transform/src https://github.com/JairusSW/as-object/tree/master/transform/src

If you'd like to talk, I'm also interested in implementing this. You can email me at me@jairus.dev

[JairusSW]

You could iterate over all the functions via the AST and set node.exported = true (or whatever the property is on a FunctionDeclaration)

If I were you, I would have the option to denote a certain function to be exported using a decorator like @export function foo(): void {}

[CountBleck]

has an option to auto generate a json file with info on all classes

Yeah, reflection seems to be a necessity whenever serialization (as-json, ASON) or introspection (as-pect, as-object) is involved... I'm wondering how that'd work with generics and such. Also, generating a JSON file might not enough for a lot of cases, because code might need to be generated based on that data... @JairusSW wdyt?

ways to externally call any function

That wouldn't be an ideal thing to enable by default, because that would stop inlining and other optimizations. I don't think that'd work too well with generics either.

[CountBleck]

@Mudloop also, what's your use case for AS?

[JairusSW]

has an option to auto generate a json file with info on all classes

Yeah, reflection seems to be a necessity whenever serialization (as-json, ASON) or introspection (as-pect, as-object) is involved... I'm wondering how that'd work with generics and such. Also, generating a JSON file might not enough for a lot of cases, because code might need to be generated based on that data... @JairusSW wdyt?

ways to externally call any function

That wouldn't be an ideal thing to enable by default, because that would stop inlining and other optimizations. I don't think that'd work too well with generics either.

I'd operate directly on the AST data and then leave it up to the bindings generator

[Mudloop]

Thanks for all the responses!

@Mudloop also, what's your use case for AS?

I'm actually building a little game engine with an electron-based editor, and I want to be able to have game/runtime code in webassembly, and have the editor will manipulate the memory, for things like an undo system. The editor will work directly on the wasm memory, but using typescript (js) classes. Doing "new Vector2()" in the editor environment would automatically create one in the wasm instance, and doing "v.x = 10" would update it there.

That allows me to have web components for inspector panels etc, which are automatically synced with the game data.

I got pretty far with the Transform based parsing/modding, and am able to instantiate linked objects from outside of the wasm instance. But generics are kicking my behind.

So I'm switching gears and decided to manipulate the WAT file instead, those are pretty trivial to parse, and seem to have all the info I need. I can easily add exports for all functions, and I think I can even automatically make it mark things dirty when something changes, just need to mod the generated "#set" functions for fields.

Haven't tested the new approach yet, only made a quick and dirty WAT parser, and made it add some exports, but haven't tested if that actually works as intended.

[Mudloop]

That wouldn't be an ideal thing to enable by default, because that would stop inlining and other optimizations. I don't think that'd work too well with generics either.

Yeah, definitely wouldn't suggest it being enabled by default. And indeed, generics make it tricky. But the generic instances of all (used) functions are in the .wat file, hence my new approach of manipulating that directly. I'm still grabbing some info with a custom Transform.

[Mudloop]

Hey, @Mudloop Here's a few resources that might help you https://github.com/JairusSW/as-json/tree/master/transform/src https://github.com/JairusSW/as-object/tree/master/transform/src

If you'd like to talk, I'm also interested in implementing this. You can email me at me@jairus.dev

This is quite useful. One thing I’m curious about is whether your implementation somehow makes sure decorated classes are present in the build.

I haven’t found a way to ensure a constructor exists if the class isn’t ever instantiated in the code. But when dealing with serialisation / asset loading, it’s entirely possible that a class is never used directly, especially if it extends a base class or implements an interface, and is only ever instantiated through serialisation.

AssemblyScript is pretty smart at culling unused stuff, but that’s a problem when you can want to instantiate things externally.

My current strategy of not touching the ast but modding the WAT instead seems to work, except for the fact that I have to add dummy code to make sure everything is included.

[JairusSW]

Hm, that's a good point. If you just declare a class, it'll be DCE'd. You could mitigate this by having an exported dummy function reference the class. As far as tables, you can get the table indices of a function using changetype<usize>(fnRef), right @CountBleck

[CountBleck]

Actually, I believe it's fn.index, but don't take my word for it. IIRC, Function is an object that's present in linear memory.

[Mudloop]

Hm, that's a good point. If you just declare a class, it'll be DCE'd. You could mitigate this by having an exported dummy function reference the class. As far as tables, you can get the table indices of a function using changetype<usize>(fnRef), right @CountBleck

Think I found a decent way to force inclusion of classes without requiring dummy code to be generated per class (or generic instance) :

export class Registry {
    static enabled: bool = false;
    static add<T>(): void {
        if (Registry.enabled) instantiate<T>();
    }
}

export class SomeClass {
    value: f32 = 0;
}

Registry.add<SomeClass>();

This "registry" doesn't ever need to be enabled (or actually register anything), it's just there to trick the compiler. It's better than a decorator because that wouldn't work with generic classes.

To force it to also include methods, I could do this :

export class Registry {
    static enabled: bool = false;
    static add<T>(callback: ((t: T) => void) | null = null): void {
        if (Registry.enabled) {
            var o = instantiate<T>();
            if (callback != null) callback(o);
        }
    }
}

export class SomeClass {
    value: f32 = 0;
    someMethod(): void {
        // ...
    }
}

Registry.add<SomeClass>((t) => {
    t.someMethod();
    // other methods
});

Edit : note that this could also be auto-populated based on decorators, but having the option to manually register classes for inclusion is useful, especially for adding types you can't decorate, like Array< SomeClass >

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in one week if no further activity occurs. Thank you for your contributions!