Apply `BindExport` to table elements as well

[alexcrichton]

Currently in the explainer it mentions the BindExport operation as how exports are modified with their WebIDL counterpart, and then describes:

When the module's exports are extracted, each webidl-bind statement binding an export must invoke BindExport to replace the WebAssembly export with a bound Web IDL Callback value.

When integrating the current explainer into the wasm-bindgen project I've found though that in our implementation of callbacks we also need to have WebIDL bindings for table elements. For example we'll use table elements in JS for passing Rust-defined closures to JS, where JS would otherwise call the equivalent of wasm_exported_function_table.get(the_index)(...). In this case we'll want to leverage WebIDL bindings for efficiently passing arguments like strings back and forth.

Currently @fitzgen has a strawman ast format for WebIDL bindings which does specifically allow for this in the technical sense, but it seemed from rereading the explainer that this wasn't originally considered.

I don't think it'd modify the proposal too much, but it might be good to explain that BindExport is called for elements of function tables in a module whenever a function is accessed through that (not just through an export item)

[lukewagner]

This is a great question. So one important detail about the current proposal is that BindExport takes a funcref value (which, by definition, has a core-wasm signature) and produces a Callback value (which, by definition, has a Web IDL signature, is distinct from the original funcref value (in the same way that f.bind() is distinct from f in JS) and is not a funcref itself (having no core-wasm signature)).

Thus, the result of BindExport can't be stored in a Table<funcref>, but would need to be stored in a Table<anyref>. I don't see any way that easily falls out of the current design to initialize a Table<anyref> with bound function elements.

For the moment, the way I'd achieve your goal is to import WebAssembly.Table.prototype.set with a Web IDL signature (any, i32, Callback) -> void which would allow you to dynamically convert funcrefs to Callbacks with the bind-export operator. (Which is admittedly goofy, so we should study this problem some more and see if there is a general principled solution here.)

[fgmccabe]

The 'soap lambda' approach would have no problem with this. Its a special case of allowing constants.

On Thu, Jun 6, 2019 at 1:41 PM Luke Wagner notifications@github.com wrote:

This is a great question. So one important detail about the current proposal is that BindExport takes a funcref value (which, by definition, has a core-wasm signature) and produces a Callback value (which, by definition, has a Web IDL signature, is distinct from the original funcref value (in the same way that f.bind() is distinct from f in JS) and is not a funcref itself (having no core-wasm signature)).

Thus, the result of BindExport can't be stored in a Table, but would need to be stored in a Table. I don't see any way that easily falls out of the current design to initialize a Table with bound function elements.

For the moment, the way I'd achieve your goal is to import WebAssembly.Table.prototype.set with a Web IDL signature (any, i32, Callback) -> void which would allow you to dynamically convert funcrefs to Callbacks with the bind-export operator. (Which is admittedly goofy, so we should study this problem some more and see if there is a general principled solution here.)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/WebAssembly/webidl-bindings/issues/35?email_source=notifications&email_token=AAQAXUCWK3ABB6BDAE5V6D3PZFZAFA5CNFSM4HVISGG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXEDFDI#issuecomment-499659405, or mute the thread https://github.com/notifications/unsubscribe-auth/AAQAXUE4ETWE6LLRHGXLPZLPZFZAFANCNFSM4HVISGGQ .

-- Francis McCabe SWE

[alexcrichton]

Hm I'm not sure I fully understand, so to say it back to you @lukewagner the idea is to have a scheme like:

• For each signature of callback, create a wasm function that does:

(func (export "some_unique_name_based_on_the_signature") (param i32) (result funcref)
  local.get 0 
  table.get $the_function_table)

• Add a webidl binding for some_unique_name_based_on_the_signature where the the return value uses the bind-export operator to turn the funcref into a typed WebIDL function.

• Don't use table.get(...) in JS, but instead call some_unique_name_based_on_the_signature with the index we want and then that'll materialize a magical WebIDL binding value.

Is that roughly what you were saying (ish)? I didn't quite follow why we'd import Table.prototype.set and why we'd store these values into a Table<anyref> (after loading them from a funcref? (or are you thinking that the Table<anyref> would be initialized at startup and then it's not re-manufactured each time?)

@fgmccabe ah sorry I don't know what a "soap lambda" is :(

[fgmccabe]

Its part of the discussion around the ongoing effort for host/web/xx bindings

It's a simple idea: say you are trying to access an external function whose signature took the form: ImportFn:(Person)=>Address wasm does not understand this directly. But given a suite of declaratively specified operators that understand wasm and the IDL we can wrap this imported function as:

(mLoc) => unwrap

(ImportFn(wrap(mLoc)))

(The actual operators are made up here, the idea is to have a whole bunch of them) This function is directly callable from wasm; and it 'calls' the imported function using the type language for that function. The embedder's role is to understand the unwrap/wrap operators -- together with a complementary set of operators used by the publisher of the API -- to generate the actual call to the actual function that supplies a person's address.

On Thu, Jun 6, 2019 at 2:25 PM Alex Crichton notifications@github.com wrote:

Hm I'm not sure I fully understand, so to say it back to you @lukewagner https://github.com/lukewagner the idea is to have a scheme like:

• For each signature of callback, create a wasm function that does:

(func (export "some_unique_name_based_on_the_signature") (param i32) (result funcref) local.get 0 table.get $the_function_table)

-

Add a webidl binding for some_unique_name_based_on_the_signature where the the return value uses the bind-export operator to turn the funcref into a typed WebIDL function.

Don't use table.get(...) in JS, but instead call some_unique_name_based_on_the_signature with the index we want and then that'll materialize a magical WebIDL binding value.

Is that roughly what you were saying (ish)? I didn't quite follow why we'd import Table.prototype.set and why we'd store these values into a Table (after loading them from a funcref? (or are you thinking that the Table would be initialized at startup and then it's not re-manufactured each time?)

@fgmccabe https://github.com/fgmccabe ah sorry I don't know what a "soap lambda" is :(

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/WebAssembly/webidl-bindings/issues/35?email_source=notifications&email_token=AAQAXUBLM5V7GL527XADESLPZF6C5A5CNFSM4HVISGG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXEGSFA#issuecomment-499673364, or mute the thread https://github.com/notifications/unsubscribe-auth/AAQAXUDR5Z5H47LLDLYL7H3PZF6C5ANCNFSM4HVISGGQ .

-- Francis McCabe SWE

[fgmccabe]

It's called 'soap lambda' because it is reminiscent of how soap molecules work in cleaning

On Thu, Jun 6, 2019 at 2:49 PM Francis McCabe fgm@google.com wrote:

Its part of the discussion around the ongoing effort for host/web/xx bindings

It's a simple idea: say you are trying to access an external function whose signature took the form: ImportFn:(Person)=>Address wasm does not understand this directly. But given a suite of declaratively specified operators that understand wasm and the IDL we can wrap this imported function as:

(mLoc) => unwrap

(ImportFn(wrap(mLoc)))

(The actual operators are made up here, the idea is to have a whole bunch of them) This function is directly callable from wasm; and it 'calls' the imported function using the type language for that function. The embedder's role is to understand the unwrap/wrap operators -- together with a complementary set of operators used by the publisher of the API -- to generate the actual call to the actual function that supplies a person's address.

On Thu, Jun 6, 2019 at 2:25 PM Alex Crichton notifications@github.com wrote:

Hm I'm not sure I fully understand, so to say it back to you @lukewagner https://github.com/lukewagner the idea is to have a scheme like:

• For each signature of callback, create a wasm function that does:

(func (export "some_unique_name_based_on_the_signature") (param i32) (result funcref) local.get 0 table.get $the_function_table)

-

Add a webidl binding for some_unique_name_based_on_the_signature where the the return value uses the bind-export operator to turn the funcref into a typed WebIDL function.

Don't use table.get(...) in JS, but instead call some_unique_name_based_on_the_signature with the index we want and then that'll materialize a magical WebIDL binding value.

Is that roughly what you were saying (ish)? I didn't quite follow why we'd import Table.prototype.set and why we'd store these values into a Table (after loading them from a funcref? (or are you thinking that the Table would be initialized at startup and then it's not re-manufactured each time?)

@fgmccabe https://github.com/fgmccabe ah sorry I don't know what a "soap lambda" is :(

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/WebAssembly/webidl-bindings/issues/35?email_source=notifications&email_token=AAQAXUBLM5V7GL527XADESLPZF6C5A5CNFSM4HVISGG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXEGSFA#issuecomment-499673364, or mute the thread https://github.com/notifications/unsubscribe-auth/AAQAXUDR5Z5H47LLDLYL7H3PZF6C5ANCNFSM4HVISGGQ .

-- Francis McCabe SWE

-- Francis McCabe SWE

[fitzgen]

Currently @fitzgen has a strawman ast format for WebIDL bindings which does specifically allow for this in the technical sense, but it seemed from rereading the explainer that this wasn't originally considered.

Small clarification: that linked code snippet is equivalent to binding to a function via a static index in the function index space, not via a dynamic index into a function table.

Since funcref is not really a thing yet, I would be in favor of minimizing our dependencies on other proposals and having variants of bind that work with both static indices in the function index space and dynamic indices into a function table. The latter would presumably need some sort of binding operator to pluck the dynamic index out of the outgoing wasm value tuple.

[lukewagner]

@alexcrichton That's a good way to check understanding :) No, not like that: iiuc, you are trying to set up a table of bound functions (presumably so that later they could be accessed and called at runtime, without needing to create a fresh binding each time). For that purpose, what you'd import and bind to is Table.p.set. The binding for the "new value" argument of Table.p.set would use the bind-export binding operator to create the Callback value that is written into the Table<anyref> (which you could later call from JS via tbl.get(i)(...), just like now.

(To wit, I realized that you don't actually need to import Table.prototype.set; you could just import any identify function, using the binding on the parameter to wrap the funcref into a Callback, which can then be returned back to wasm as an anyref and stored in a Table<anyref> via wasm's table.set.)

@fgmccabe I don't think the "soap lambda approach" is fundamentally different than the explainer as written other than terminology; if you accept the basic constraint that we're not changing core wasm semantics, and thus describing declarative conversions as things core-wasm calls as imported functions, one has the same problem and roughly the same space of solutions as I'm describing (assuming I've understood @alexcrichton's problem statement correctly).

[alexcrichton]

@lukewagner ah ok so if I understand you right this time around we would need a third table (one for the LLVM function table, one for the anyref heap/stack, and one for typed function table exports). This third table would be populated at runtime by first acquiring a funcref to a wasm function, passing that out to JS and getting it back as a bound function (via WebIDL bindings and bind-export), and then we'd store that into the third table. Later when JS actually invokes these closures it invokes closures loaded from the third table, not tables loaded from LLVM's generated table. Is that right?

If so that does introduce a dependency on the funcref proposal which is somewhat unfortunate (as @fitzgen was mentioning). It's also sort of unfortunate that it needs runtime instantiation which we'd have to arrange to happen at the right time :(

[lukewagner]

@alexcrichton Right. I think we can definitely improve on this, we just need to collect some more use cases to find out what's the general problem and what the "right" fix is. For example, just as core wasm has "elem" sections that can be used to efficiently initialize tables (moreso than ref.func+table.set), maybe Web IDL Bindings would have its own "bound elem" sub-section that does likewise, as a more efficient way to do (ref.func+bind-export+table.set)?

[alexcrichton]

Sounds plausible to me! I think I at least understand the problem space here better now :)

[alexcrichton]

FWIW in a recent refactor to use WebIDL bindings internally, @lukewagner your proposed idea worked out great. The auxiliary data was basically a list of new bindings which can be called, and for now each of these bindings is simply a JS shim that does the transformation and calls the underlying table function.

[pchickey]

Closing as resolved - the concerns initially raised are no longer relevant after the pivot into Interface Types