[Proposal] Assembler Directives

tjhancocks commented 4 years ago

Aspect	Information
Author	Tom Hancocks
Proposal Date	12 December 2019
Status	In Progress / Writing

Proposal Outline

At present KDL is a largely static definition language, which is not ideal long term. It requires type information to be baked into the assembler, and thus new versions to be deployed in order to keep pace with the Kestrel engine. It also makes it impossible for forks, variants of Kestrel to add new types and make use of KDK.

By adding Assembler Directives we can allow for more dynamism and the following things to happen:

Metadata Functions
External KDL Imports
Variables
Looping
Type Definitions

Detail

KDL already has the basic syntax for Assembler Directives built in.

@directive { 
   ... do something here ...
}

Directives start with an @ and the directive name. Presently only the @out directive exists, and it is used for dumping information to the standard output. It might be used for something like:

@out {
    "AwesomeTech is designed for use in EV Nova, and developed by John Smith."
}

It can provide a means of informing users about the plugin/data file being assembled (presuming the plugin/data file has been distributed as source.)

There are 4 primary extensions to Assembler Directives that will be discussed here.

§1 - Meta Data Functions

It could be made possible for meta data to be encoded into plugins, either as extended file attributes, version resources, or some other yet undefined resource type.

For example, assume the following directives are used in the source for the AwesomeTech plugin.

@require { "Nova" } ` Takes the name of the scenario it requires
@name { "AwesomeTech" }
@author { "John Smith" }
@version { "1.0" }

These directives are an example of what could be added. Both @name, @author and @version are self-explanatory and obvious, but @require warrents further explanation.

Assuming the addition of an option to the assembler that allows referencing/linking against an existing scenario, the @require directive could be used to enforce the need to be linking against a specific scenario.

As an example, presume the following invocation of the assembler

$ kas --scenario="/Applications/EV Override.app/Contents/Scenario" AwesomeTech.kdl

Presumming the Scenario defines the following

@name { "Override" }

And the plugin specifies the following

@require { "Nova" }

Then the assembler can infer that it will be referencing an incorrect set of information, and thus warn the user.

§2 - External KDL Imports

It is rarely a desirable thing to include absolutely all of your code into a single file, and thus the ability to import other code files into the current context is important. At present KDL provides no way to do this.

There are multiple ways this will likely work ultimately. This is one of the mechanisms that will likely be required.

@import { "/path/to/type-definitions.kdl" }

This is using the same familiar directive syntax, but with a slightly different mode of operation. All directives execute immediately upon being seen. The @import can utilise this behaviour, open the specified KDL file, perform lexical analysis of the contents and insert the resulting tokens into the current semantic analysis context in place of itself.

If the imported KDL script includes type definitions, then those types will now be seen and parsed by the current semantic analysis context, and become available to use in the script.

` Import the Mission resource type definition.
@import { "/path/to/game/mission-definition.kdl" }

` Use the Mission resource type.
declare Mission {
    ...
}

§3 - Variables

to be spec'd

§4 - Looping

to be spec'd

§5 - Type Definitions

It should ultimately be the responsibility of a scenario to define the resources it needs. This allows for appropriate naming to be used within KDL, and easy expansion and reuse of the assembler.

Before getting started this is an example of what a type definition might look like.

@define {
    name = "Sprite Animation";
    field("sprites") {
        required;
        value(type = resource_reference, offset = 0);
    };
    field("masks") {
        deprecated("This field is not used by Kestrel.");
        value(type = resource_reference, offset = 2);
    };
    field("size") {
        required;
        value(name = "width", type = integer, offset = 4, size = word);
        value(name = "height", type = integer, offset = 6, size = word);
    };
    field("tiles") {
        required;
        value(name = "x", type = integer, offset = 8, size = word);
        value(name = "y", type = integer, offset = 10, size = word);
    };
}

The definition needs to be able to specify the locations of values in the binary data, and the sizes of those values. To begin with the definition language should be simple in its capability, and extended later. This should allow focusing on getting a working system in place first.

Value Types

The following value types should be specifiable in definitions:

resource_reference: This value expects a resource id. Ideally there should be a way to constraint what happens when a file("...") value is given as a reference.
integer: This value expects an integer. If an integer type is specified then a size must be specified as well (byte (1), word (2), dword (4), qword (8)).
string: This value expects a string to be provided. If the string type is specified then a length must be specified as value.
c_string: This value expects a variable length, null terminated string to be provided.
p_string: This value expects a variable length string with a maximum length of 255. The length is encoded in a byte at the start of the string.
color: This value expects an RGB color encoded as a dword integer.
bitmask: An integer type that merges symbol values together. If an bitmask type is specified then a size must be specified as well (byte (1), word (2), dword (4), qword (8)).

Symbols

Symbols are a way of specifying a semantically appropriate representation of a value. For instance, take the following resource declaration.

declare Person {
    new(id = #130, name = "Nameless") {
        ` ... stuff ...
        government = none;
        ` ... stuff ...
    }
}

Rather that specify the value -1 directly, we have provided the symbol none. The assembler knows to subsitute the none for -1 in this context.

Symbols can be defined by doing the following.

field("foo") {
    value(type = bitmask, offset = 0, size = word) {

    }
};

Community

Has this proposal being discussed with the community? Changes to KDL affect everyone, and thus the more people that have been involved in refining it the better. If you haven't discussed it yet, then perhaps consider taking it over to the Discord and discussing it there first.

Alternative Solutions Considered

Detail some alternatives that were considered, and why they would not really solve this particular problem.

andrews05 commented 4 years ago

Should the type definitions be implemented first so that we don't have to create all the assemblers?

tjhancocks commented 4 years ago

I'm not sure.

On the one hand it makes sense that these are done first, but on the other hand should KAS have built in backups/defaults to use. I'd probably lean towards wait...

andrews05 commented 4 years ago

Perhaps the standard type definitions could be compiled into the assembler as defaults.

tjhancocks commented 4 years ago

Maybe 🤔

How would that look though? Are we talking embedded the actual definitions as KDL into the assembler, or auto generating C++ code based on the definitions. Neither sound pleasant to me, and one sounds horrific.

But I'm not opposed to this solution if any one has a better idea than those.

andrews05 commented 4 years ago

I was thinking the former, so it would have to interpret them at runtime. (Otherwise you'd need to build the assembler, use it to generate code from the type definitions, and then rebuild it... yeah, horrific). Surely this would be a lot less work than implementing all the assemblers?

tjhancocks / kestrel-development-kit