WIP: an RFC for the pre-build strategy

[dherman]

This RFC proposes a strategy for making Neon pre-builds feasible for libraries. This way Neon users can ship libraries whose downstream consumers can be completely unaware of the use of Rust and Neon. In other words, it makes it more tractable to ship native libraries written in Neon with zero requirements for what software their consumers have installed on their machine.

Rendered

[kjvalencik]

NAPI is stable in v10, which becomes LTS this fall. Is this another viable alternative or just something that might ease the implementation?

If there are features in neon that currently cannot be supported by NAPI, what about moving them behind a feature flag?

[dherman]

@kjvalencik These are really good questions. You're right that it's about time for us to start figuring out what our approach to NAPI is going to be. So I guess here are some of the questions we have to answer:

1. How much do we care about missing support for core JS types like RegExp and Date?
2. Does napi_create_async_work etc give us all the same access to the libuv thread pool as the current task API and @tptee's threads RFC need?
3. Do we need a transition strategy until 10 becomes LTS?
4. Can we implement all the Neon class functionality with N-API?

I bet RegExp etc is not particularly important, and if it is important, we can advocate for putting them into N-API.

I bet the async stuff gives us everything we need to make the UV threadpool available, but I'll investigate.

For transition strategy, my guess is all of this will take enough time that the question will become moot, but we can discuss tactics if/when necessary.

Classes might be a bigger challenge. I guess @matklad doesn't really like the class feature, but I do. :) Let me dig around and see what that would entail.

[dherman]

I should say, @matklad made good points, and if the only thing blocking us adopting N-API was that the classes design doesn't fit it, I'm definitely open to revisiting the classes design. The more fundamental feature that we have to support one way or another, though, is the ability to let a JS object own a Rust data structure.

[dherman]

OK so the place to look is the object wrapping APIs. I'll keep investigating!

[tptee]

I think napi_create_async_work is the equivalent of uv_queue_work? So that’d cover the existing Task implementation using the libuv threadpool. It looks like these functions cover async handles (what’s needed for the Rust thread RFC): https://nodejs.org/api/n-api.html#n_api_custom_asynchronous_operations

[dherman]

Here's my proposed course of action: we know we need to explore the N-API backend regardless, and since it sounds like N-API's ABI stability might make for a simpler story than dynamic linking the Neon runtime, we should:

• reject this RFC for now
• start working on an N-API backend
• revisit this RFC's idea only if N-API doesn't solve the problem of large build matrices for some reason

Seem reasonable?