Extension and Capabilities Querying System

[FlyingJester]

I suggest a system be put in place that allows scripts to query the capabilities and extensions available from the engine they are running under. This would be exposed as a function, for instance sphere.getExtensions(), that returns and array of strings which describe the capabilities of the engine. I propose the following structure for the capabilities strings: sphere_[authority]_[description] The authority part of the string is similar to ARB/EXT/NV/ATI/AMD in OpenGL capabilities strings, and represents what level of support this capability has. It would be omitted in full engine support (for instance, it's not engine-dependant per se to support MIDI files, but not all engines may do so), while engine-dependant functionality will have the engine's prefix as the authority (for instance, TurboSphere supports true-type fonts, while it is unlikely this will ever be a full API requirement, and so it would be under ts authority).

Legacy function from the older engine could also be specified with this, perhaps with the authority legacy.

For example:

• sphere_midi_sound_object
• sphere_array_buffers
• sphere_legacy_complex_primitive
• sphere_legacy_direct_surface_blit
• sphere_ts_ttf_font_object
• sphere_ts_external_soundfont
• sphere_amd_bonjour_networking

This would put codified names to abilities that are optional to an engine, as well as allow non-standard extensions to be easily tested for. If we are going to have more than a single, unified engine, having the ability to ask about the engine (beyond just what its version number is) is important. It also helps quite a bit for plugin-based engines, where even knowing all the intricacies of the engine's version tells you very little about what is available.

The core, required engine capabilities can still be queried with the equivalent of GetVersion(), which really should be (at this point) returning the API version rather than a true engine version number.

[joskuijpers]

As you have already implemented this in TS, could you Clone && make the changes in engine.js && push? After all, GitHub allows for multiple people changing code :smile:

[FlyingJester]

A big reason I don't push here that often is that I can't check that my changes actually work in jsdoc. I can't run Node.js on my dev machine since its V8 conflicts with TS's.

I'll write it up, though. Or at least attempt to :)

[joskuijpers]

Oh no biggie. Not all of my code works well either. So just edit it, and I'll fix it up later :)

[fatcerberus]

I'm not sure I like the underscores in the extension names. In minisphere I implemented GetExtensions() to return strings in this format:

• sphere-legacy-api
• sphere-obj-constructors
• sphere-galileo
• set-script-function

And so forth. Using dashes makes names easier to type and less error prone as you don't have to reach for the shift key while typing out extensions.

[FlyingJester]

I suggested underscores for two reasons:

• This lets the tokens be actual identifier names
• This is how the OpenGL extension system works, which is what gave me the idea in the first place.

[fatcerberus]

Is there any benefit to allowing them to be used as identifier names, though? In JavaScript you have the index syntax obj["string"] which allows any arbitrary string to be used as a key, so the point seems moot.

[FlyingJester]

I originally planned on implementing any extension in JS within an object of that name. Another way to check for an extension would be

if (typeof sphere_authority_extension != "undefined") /* ... */

I personally don't have too strong a preference, but I don't really think using a dash is less error prone. I do find underscores a bit easier to read, though.

...cool bikeshed, btw :)

[fatcerberus]

Bikeshed...? Huh?

Ugh, extensions as global objects... I'm glad you didn't implement that idea. :P It's so much nicer to just call GetExtensions() and run .indexOf() on the array (or Link, in my case) to check for the relevant extension(s). Here's my current engine check in Specs:

var extensions = typeof GetExtensions !== 'undefined' ? GetExtensions() : [ 'sphere-legacy-api' ];
var q = Link(extensions);
var isSupportedEngine = GetVersion() >= 1.5
    && q.contains('sphere-legacy-api')
    && q.contains('sphere-obj-constructors')
    && q.contains('sphere-obj-props')
    && q.contains('sphere-galileo')
    && q.contains('sphere-new-sockets')
    && q.contains('set-script-function');
if (!isSupportedEngine) {
    Abort("This engine is not supported.\n");
}

I could indeed refactor that to use the typeof method (without even adding any lines of code), but it wouldn't be as clear what I'm checking for then. Here it's obvious that I'm calling GetExtensions() and then checking to see whether the return contains the relevant strings. Just testing whether extension objects exist apropos of nothing loses clarity, in my opinion.

[FlyingJester]

The extensions as global objects would solely have been for the implementation. However, this is not really useful anymore since all functionality has proper namespaces anyway.

Bikeshed...? Huh?

Dashes or underscores is such a tiny detail. It's a bikeshed problem (what color do we paint the bikeshed?).

[fatcerberus]

Bikeshed problem or not, we're trying to design a unified standard here, so we do have to end up coming to an agreement one way or the other. :sweat_smile:

[fatcerberus]

Getting away from splitting hairs: I propose we standardize GetExtensions() (or sphere.getExtensions() if that's what we go with, although I'm not a fan of such low-level functionality being namespaced/modularized) such that the very first extension always names the engine in use. For instance, minisphere or sphere-sfml.

See, because here's what I'm thinking: We can standardize the API all we want, but realistically there are always going to be discrepancies due to bugs, subtle behavior differences not covered by the spec, etc. So this way, a game can simply query GetExtensions()[0] to get the engine designation and adjust its logic accordingly.

[joskuijpers]

I think that is fine, but using arrays seems error prone, as the order of extensions is not predefined. If there is a, b and c, c is [2]. If b is missing, c becomes [1]. Why not use an object with keys being the ext names? Then a ['myext'] does the job.

For namespacing the low level functions: do it. You should minimize the number of global functions. The module system provides a context for every file (bot sure how far i got with the module system), and it will need to provide every global object you need in a file. This will only be like 4 or 5 standard items, one being 'sphere'.

On Apr 25, 2015, at 8:40 AM, Bruce Pascoe notifications@github.com wrote:

Getting away from splitting hairs: I propose we standardize GetExtensions (or sphere.getExtensions() if that's what we go with, although I'm not a fan of such low-level functionality being namespaced/modularized) such that the very first extension always names the engine in use. For instance, minisphere or 'sphere-sfml`.

See, because here's what I'm thinking: We can standardize the API all we want, but realistically there are always going to be discrepancies due to bugs, subtle behavior differences not covered by the spec, etc. So this way, a game can simply query GetExtensions()[0] to get the engine designation and adjust its logic accordingly.

— Reply to this email directly or view it on GitHub.

[fatcerberus]

I know that. I'd expect most people would use either Array:indexOf or a query library like @Radnen's Link to check for specific extensions; what I'm proposing is to standardize the location of the engine designator to the first slot, this way games can easily check which engine they are running on without having to add an additional API for it.

I don't see that the order of the rest of the extensions being variable is an issue; anyone checking for a specific extension at, say, GetExtensions()[2] needs to be shot.

As for the namespacing, I still have an issue with it. I despise it for the same reason I despise C++ putting the entire standard library in the std namespace--which I invariably end up importing with using namespace std; anyway. But that's a topic for another issue, so let's not discuss this further here. :-p

[fatcerberus]

As I experiment with the CommonJS module system more, I'm beginning to feel like a global extension registry is unnecessary and maybe even counterproductive. If you try to require() a module and it fails, the require call will throw, which allows you to catch the exception and gracefully degrade in the face of missing modules. If it's important to know what functionality is exposed by a module at a more granular level, then the module should expose that itself, either through a version property or for more complicated modules like the graphics module, a module-specific extension list.

In the end the engine should merely be a facilitator, not an assassin gatekeeper. A GetExtensions()-like system forces all modules to check in with the extension manager, which works against the goal of modularity in my opinion.

[FlyingJester]

The point of the extension system was mainly to explicitly state certain features of the environment, not to force plugins or modules to register. It was more intended to allow different implementations some leeway in how they implemented certain features, and to allow features to easily be optional on other platforms, but defining the bare minimum as the standard and optional features as extensions.

For instance, one implementation might support loading an Image straight from a filename, while another might require a Surface intermediate. Or it would allow you to query which version of GLSL, HLSL, or some other shader language was available.

[fatcerberus]

Okay, that makes more sense then. Some optional functionality can only be provided by the implementation, or may not even have an API attached to it (like the GL_texture_non_power_of_two example you used somewhere else), but it's still useful to be able to advertise its presence.

[joskuijpers]

What Martin says is correct, that is indeed how we designed the extension system, purely for the implementation of the game system.

The module system is for libraries such as Bluebird or Link.

(Speaking ofbluebird, is everything asynchronous yet? We should use promises!!!!)

On Apr 24, 2016, at 5:48 AM, Bruce Pascoe notifications@github.com wrote:

Okay, that makes more sense then. Some optional functionality can only be provided by the implementation, or may not even have an API attached to it (like the GL_texture_non_power_of_two example you used somewhere else), but it's still useful to be able to advertise its presence.

— You are receiving this because you commented. Reply to this email directly or view it on GitHub

[fatcerberus]

Ugh, no. minisphere includes promises - see the miniRT/pacts module. But there are no asynchronous APIs other than the Async() function to dispatch scripts to run in the next cycle. I'm not a big fan of async-everywhere systems. The one time I tried to write something using node.js I ended up wanting to kill someone.

Granted a lot of that has to do with JS syntax - if JS had something like C# async/await and events I think I might enjoy it more. That said, async code can be a nightmare to debug, and I don't think it buys us anything in a game engine regardless.

[joskuijpers]

I just want to try it out. Just make some code and see what happens. I am mostly concerned about file reads and write. Things that really block. The rest of the stuff should be near-real time, because it is a game engine. IF we do Async for stuff like files, I would suggest it uses Promises.

[fatcerberus]

This feature is implemented in minisphere 4.0 as system.extensions. To check for presence of an extension, a game can do:

// writing save file
if (system.extensions.sphere_stateful_rng) {
    // save RNG state in save file
}

I went with a somewhat simpler naming convention than described above of [authority]_[description], where authority is sphere for widely supported extensions, and (usually) an engine name for vendor-specific ones, e.g. minisphere_ssj_api. This is more readable I think than just tacking sphere_ and an abbreviation onto everything indiscriminately.

[fatcerberus]

minisphere 4.1 advertises the following extensions:

• sphere_glsl_shader_support
• sphere_stateful_rng - RNG.fromState() and state property
• sphere_v1_compatible_api - Sphere 1.x compatibility
• minisphere_ssj_api - SSJ interface, minisphere specific

[fatcerberus]

Question: Do we want all core features to have their own extension, e.g.:

• sphere_async_dispatch
• sphere_stateful_rng
• sphere_classic_api
• etc.

Or would it be better to define a core set of functionality and then only optional features get an extension string?