joneubank commented 3 years ago

SQON and SQONMixer

Proposal for Synthetic SQON Management Library

SQON Builder API Updates

The SQON portion of the library has been refactored to no longer follow the builder pattern. This has been published as sqon-builder version 2.0.0 . Updates to this portion of the API are available in the README .

Of note, we simplified some of the terminology of the two operator types that combine into a SQON. These operators are now known as Combinations and Filters. SQONs support and, or, and not combinations, and in, gt, lt filters.

The SQON class was also given a toString() method as a convenience to serialize the data into a string. This is just a convenience wrapper around JSON.stringify(sqon). Any SQON can be passed directly into an http request library (axios or fetch for example) without the need to .build() it anymore.

SQONMixer

The task being accomplished by "SyntheticSQONs" can be summarized as maintaining a list of reference SQONs and other combinations of these references, and then in the future being able to resolve those references into a SQON usable in a query.

To simplify terminology and separate the functionality of the SQON Builder library from the existing Arranger implementation, we're proposing the introduction of a SQONMixer that will perform these tasks.

Basic Use Case

The basic usage of this object would be adding multiple SQONs, then using the SQONMixer to combine them, and finally resolve the combined entry into a new SQON.

import SQON, { SQONMixer } from "sqon-builder";

const mixer = new SQONMixer();

const sqon1 = SQON.in("name", ["Jim", "Bob"]);
const sqon2 = SQON.gt("score", 9000);

const mix1 = mixer.put(sqon1);
const mix2 = mixer.put(sqon2);
const mix3 = mixer.or([mix1, mix2]);

const combinedSqon = mixer.resolve(mix3);

The value of the combinedSqon would be:

{
  "op": "or",
  "content": [
    {
      "op": "in",
      "content": {
        "field": "name",
        "value": ["Jim", "Bob"]
      }
    },
    {
      "op": "gt",
      "content": {
        "field": "score",
        "value": 9000
      }
    }
  ]
}

Similar to a SQON, the mixer can combine SQONs and References via and, or, and not combinations.

The output of the mixer operations is a SQONMixerEntry, and these same combinations and resolution can be performed on these entries. For example:

// To combine two existing entries, either of the following work:
// const entry3 = mixer.and([entry1, entry2]);
const entry3 = entry1.and([entry2]);

// To resolve the reference:
// const resolvedSqon = mixer.resolve(entry3);
const resolvedSqon = entry3.resolve();

Other Important Functions (Modifying Existing Entries)

Removing an Entry

Equivalent to removeSqonAtIndex

mixer.remove(entry);
// Can also be removed by ID

Removing an entry from the SQONMixer will remove that entry from the list of content in the mixer, and will loop through all other entries to remove references to it. Nothing is returned from this method. The Mixer Entry object will not be modified but it will no longer be usable within the mixer.

Updating Combination

Equivalent to changeCombineOperator

entry = mixer.changeCombination(entry, CombinationKey.And);
// OR
entry = entry.changeCombination(CombinationKey.And);

CombinationKey is a Typescript enum with values And, Or, and Not

Changes the op value, combination operation, at the root of the SQON referenced in the entry.

Circular References

Expansion on isIndexReferencedInSqon

See next section pertaining to loops.

Updating SQON Content

Alternative to removeContentFromSqon

At time of writing there isn't a clean and understandable scheme for modifying parts of an existing SQON, but updates are still required to make this system useful.

In place of modifying individual clauses of a SQON's content, it is instead proposed that the SQON held within a SQONMixer Entry can be wholely replaced.

mixer.updateSqon(entry, newSqon);

To replace an entry's content with a new SQON that references other SQONMixer Entries from this mixer, you can create the content with the .combine() method:

const referenceSqon = mixer.combine(CombinationKey.Or, [entry1, entry2]);
mixer.updateSqon(entry3, referenceSqon);

Finally, from a SQONMixer Entry, entities can be added or removed from the current content list:

entry5.add([entry1, entry3]);
entry5.remove([entry2]);

Concerning Circular References (Loops)

Modifying an existing entry by adding another entry to it can create a loop. This will happen if the first entry is in the list of references of the one being added. To prevent this, all additions to existing entries will check for reference loops before executing, and if there is an issue an error will be thrown instead of updating the entry.

To accomplish this, each entry maintains a list of the IDs of the other entries it references in the property .references. To get the complete list of Mixer Entries that are required to resolve this entry, you can use the method .allReferences(). Finally, these tools are used by the method .hasLoops(entries: SQONMixerEntries[]) which can be called form the mixer or from any entry.

API Arguments

All SQONMixer functions that accept SQONMixerEntry objects as arguments can alternately be given the entry ID as a string. The function will handle both in the same way.

For example: mixer.resolve(entry.id) or mixer.resolve(entry) are both valid.

Other APIs where this is useful are .remove, and .get, and in any of the entries for the combination methods .and, .or, and .not

Serialization

It is necessary for the SQONMixer to be serializable, so that it can be stored somewhere as a string and then reloaded in the future.

To accomplish this, SQONMixer instances have a .serialize() method that will output the mixer as a string, including all contained SQONs and references. References to SQONs which are maintained in JS memory as references will instead be serialized with their string ID.

The SQONMixer class has a static method .parse(serializedMixer) that will read in a serialized mixer from string and provide a functioning SQONMixer instance as output. All the entries are available from the mixer via the .content preroperty, which stores all entries as an object with the keys being the entry IDs:

const mixer = SQONMixer.parse(serializedMixer);
const entries = mixer.content;

Here, entries would have a value like (output from first example in this doc):

{
  '1631756097117_0': SQONMixerEntry {
    _class: 'SQONMixerEntry',
    id: '1631756097117_0',
    _mixer: SQONMixer {
      content: [Circular],
      _refCounter: 3,
      _label: '1631756097117'
    },
    references: Set {},
    sqon: SQON { op: 'and', content: [Array] }
  },
  '1631756097117_1': SQONMixerEntry {
    _class: 'SQONMixerEntry',
    id: '1631756097117_1',
    _mixer: SQONMixer {
      content: [Circular],
      _refCounter: 3,
      _label: '1631756097117'
    },
    references: Set {},
    sqon: SQON { op: 'and', content: [Array] }
  },
  '1631756097117_2': SQONMixerEntry {
    _class: 'SQONMixerEntry',
    id: '1631756097117_2',
    _mixer: SQONMixer {
      content: [Circular],
      _refCounter: 3,
      _label: '1631756097117'
    },
    references: Set { '1631756097117_0', '1631756097117_1' },
    sqon: { op: 'or', content: [Array] }
  }
}

rosibaj commented 3 years ago

@joneubank is ther ea plan to transfer this repository/move this into arranger? If so, may want ot transition this ticket.