Backer Update August 14, 2019

paul-go commented 5 years ago

This is a continuation of #6. This issue is probably more of a wiki entry an issue. Possibly we can consider moving this information into a wiki once development starts ramping up.

Emitting Persistence Layer Abstractions (PLAs)

PLAs are JavaScript constructor functions (possibly with TypeScript definitions) that construct objects that roughly correspond to the physical representation of data persisted somewhere. The goal of good PLA design is to hide the persistence operations within the semantics of JavaScript itself, to the degree that this is reasonable.

Backer's PLA implementation uses the JavaScript new operator to create new objects that are persisted. For example:

const customer = new Customer();
// Customer object created, and stored in the persistence layer.

However, these constructor functions also support a form of prototype augmentation by calling the constructor function without the new operator:

Customer(
    // Parameters passed here are attached to
    // the Customer prototype.

    // Attaches a recurrent function to all customer objects
    on("selector", () => { .. })
);

Genesis

Previously, there was this idea of a "Genesis Truth File" that acted as a set of default types from which all (or most) Truth files derived. While this idea still exists, some changes need to be made to improve compatibility with TypeScript and JavaScript.

For starters, the previous genesis included constructors for String, Number and Boolean. These need to be switched to the lower-case variants. In order to fully support the proposed prototype augmentations technique, it must be possible to augment strings and numbers, and the upper-case variants of these are JavaScript built-ins that cannot safely been altered. The lower-case variants are vacant though (and also, this will align with the primitive string, number and boolean definitions in TypeScript).

Also, the root type from which all genesis-defined types derive is going to be called any. This is to further align the system with TypeScript (previously this was called just "Type"). Therefore, each type in Truth that derives from anything defined in the genesis will also be an any, which in most cases will be nearly all types defined in a Truth document.

Clues

Previously, there was an open proposal to add a feature to Truth known as Hash types or Plus types. The purpose of this feature was to allow types to be tagged with additional meta information, without affecting aliases that purport to fulfill a contract. For example, the following Truth code would fail to validate due to "abc" not fulfilling the contract "string, required":

required
string
/".+" : string

Type
    Name : string, required

123-456 : Type
    Name : "abc"

Plus types were therefore proposed as a way to support this use case. It operated via a "+" prefix placed on type annotations (and so it would be string, +required). Such types would not be considered as part of the type contract, but rather as associated meta data that would be available to API consumers.

However, it's possible to support this use-case in a completely different way–by applying Clues in TypeScript via the prototype augmentations feature described above. For example, if we imagine that there are a pre-defined set of clues to handle modifiers like abstract and readonly, we could have TypeScript code that looks like this:

Customer(
    Clue.abstract,
)
Customer.Name(
    Clue.readonly,
    Clue.searchable,
    Clue.editable,
    Clue.required,
)

Possible Clues With Examples

any(
    // Turns any type name that isn't a valid
    // JavaScript identifier into one, using a
    // best-guess algorithm to compute a sensible
    // name for the type within JavaScript.
    Clue.identifier
)

SomeClass(
    // Causes SomeClass to be abstract
    Clue.abstract

    // Causes SomeClass to be sealed
    Clue.sealed
)

SomeClass.property(
    // Causes SomeClass.property to be readonly
    Clue.readonly

    // Causes SomeClass.property to be emitted
    // as an ES6 set / map
    Clue.set
    Clue.map

    // Causes SomeClass.property to be emitted
    // as an array with specific features.
    Clue.array
    Clue.array.pushable
    Clue.array.popable
    Clue.array.shiftable
    Clue.array.unshiftable
)

Attaching Functionality To Types With A Clue

Clues will be valid Reflex selectors, which are passable to the on() function. This will allow for functionality to be conditionally added to types that are endowed with a particular clue. For example:

string(
    on(Clue.editable, () =>
    {
        // This would apply to all things that inherit
        // from string and have the "editable" clue.
    })
)

However, because (basically) every type in a Truth document extends from any, we're able to use this to apply specific behavior to each type in the system with a particular clue. For example:

any(
    on(Clue.editable, () =>
    {
        // Applies to (basically) all types in the system
    })
)

Computed Truth

Out in the wild, we have found some scenarios that necessitate a concept of "computed Truth". These are scenarios where writing out the Truth for a particular domain would be too complex. This usually happens as a result of a permutations problem. Imagine a situation with 1000 widgets and 1500 contraptions, where a widget W may be matchable with a contraption C in the case when W and C have certain characteristics, which is computable based on a simple algorithms.

It turns out that it's possible to represent Truth Data in TypeScript, using JSON-like object constructions. These JSON-like objects could be passed to types using the prototype augmentation feature, producing code that looks like this:

Customer(
    {
        value: [string, {
            min: 10,
            max: 20
        }]
    }
)

This would dynamically attach new Truth to the model, from TypeScript.

Circular Workflow

The above specification appears to have a circular chicken-and-egg problem. The TypeScript support files appear to depend on some generated definitions. However, the process by which these definitions are generated is actually being influenced by these same TypeScript support files.

This problem can be solved if the Backer compiler is built as a long-running, real-time streaming system that says connected to a mutating Truth source (such as a Truth code editor), and sets up some kind of FileSystemWatcher to watch for changes to the TypeScript support files. This long-running process runs the TypeScript support files in some hidden background JS context (such as a WebWorker). This context will have been augmented to silently short-circut all IO operations, as well as all asynchronous behavior. For example, setTimeout() callbacks will simply not fire, fetch() calls will never return anything, promises won't ever resolve, etc. After the TypeScript support files have been executed (which can basically be done synchronously since the asynchronous functionality will have been short-circuited), the system can perform some introspection to see how or if any prototypes were augmented, and then use this information to advise the generation of the definitions.

This design causes the Backer compiler to have a circular feedback loop. For example, imagine that two files are present in the workflow:

// File.truth

Entry : Class

// File.truth.ts

Entry(
);

const e = new Entry();

The above files would generate a definition looking something like the following:

class Entry
{

}

However, the second a clue is added in File.truth.ts:

// File.truth.ts

Entry(
    Clue.abstract
);

The definition would immediately change to look like this:

abstract class Entry
{

}

Which would cause an error to be reported in File.truth.ts, because abstract classes cannot be instantiated. And so user code would become self-programmable, but in a way that doesn't inhibit type safety.

qti3e commented 5 years ago

Thanks to this proposal things are a lot clearer now, I have some comments that I thought are worth mentioning.

Instead of introducing String, Number and Boolean as some special cases that will only work in JS we can introduce some more strong types like Uint32, Int32, Float64 and ... This way it will be possible in the future to target languages like Rust or Go, of course, all of these types are just number when we translate the code to JavaScript but introducing more types will help portability to other languages.
Personally, I don't like the current API for listening to events as it becomes a bit hard to implement, what is the return type of on(...)? what should the constructor function do with that value? But this is a lot more clear:
```
Customer
.on("some-event", () => {...})
.on("some-other-event", () => {...})
```
The proposed part for Computed Truth should be some implementation details in the compiler and be hidden inside of Customer's implementation.
Also, Clues are some implementation details as well, I'd use some sort of validation layers, both Clues and Computed Truth can be implemented the same way...

paul-go commented 5 years ago

Getting the Truth stack to work outside of the JavaScript environment isn't a goal of the project. (There's a very long explanation here).
The reason for the global on() usage is because this entire API is actually built upon the Reflex Core, so TruthTalk will gain all the Reflexive library features for free this way. I probably should have made that more clear in the spec.

3 & 4 seem a bit off base to me. Maybe I didn't make something clear enough in the spec? Computed Truth is supposed to be for situations where the ontologist writing the Truth code is finding themselves having to write too much stuff out, and a simple 3-line JavaScript function would suffice. It doesn't really have anything to do with getting into the implementation of the generated Customer object.

Truebase-com / TruthStack