Embeddings, pinging & Event Horizon retry

[joelhoisko]

Summary

Adds a new feature, Embeddings! They are similar to Projections, but they are meant to be used to event source changes coming from an external system. Check the sample for an example.

Also changes the behavior of the pinging system to be more reliable and to be ready to receive pings immediately upon connecting to the Runtime. This is to deal with a bug that was causing connections between the SDK and the Runtime to be dropped. This is a breaking behavioral change and it's related to the release of version v6 of the Runtime. You have to update to version v6* of the Runtime, older versions wont work with this release of the SDK. We've added a compatibility table for checking the supported versions.

Added

• Embeddings! You can use them inline with the withEmbeddings() method, or with the @embedding, @compare, @delete and @on decorators on classes. The embeddings can be updated, deleted and fetched from the client.embeddings property.

Changed

• The reverse call connections are now ready to start receiving pings and writing pongs immediately upon connecting.

Fixed

• Fix Event Horizon connection to actually retry if it encounters an error in the connection process.
• Pinging system should now timeout a lot less than before.

[joelhoisko]

• Adds embeddings project. Most of it is copied from projections. Embeddings are split to:
  • Builder for inline and class
  • Internal for the EmbeddingProcessor that handles the gRPC requests/responses
  • Store for handling getting the embeddings. Has a tiny builder in it too.
• The @deleteMethod() decorator is called that, because delete is a reserved keyword so it can't be used as a decorator sadly. @remove() could maybe also work? Delete is such a nice word though... @obliterate() would be nice too.
• The callback
• Made projections.Builder.OnMethodBuilder too be more generic so that embeddings can use it too.
• Add ClientProcessor to act as a level of abstraction of the ReverseCallClient. Reasoning for this, was that we used to only have the events.processing.Internal.EventProcessor for that, but the EventProcessor doesn't really fit for all things (like Embeddings). So I added a new thin
• Fixed ReverseCallClient to allow pings to come first. I also refactored the class a bit to make it a bit clearer
• Added some comments to ReactiveGrpc
• Changed from using Observable<never> to Observable<void>. The never is specified as "An Observable that emits no items to the Observer and never completes.", while we were using it as an observable that will either complete or error. The void isn't the best solution either, because that is kinda signaling that the next values are just void. What we'd want is to use something like empty, but it's deprecated and I didn't figure out how to use it (funnily empty is internally just a wrapper for never hah). But I think the problem here is also that were using observables for something where a Promise would be the way to go. But that's a bigger thing, just explaining my reasoning.
• TODO: update contracts when its released
• TODO: fix the embeddings sample
• Syntax for inline builder:

  .withEmbeddings(builder => {
  builder.register(DishCounter);
  builder.createEmbedding('0767bc04-bc03-40b8-a0be-5f6c6130f68b')
      .forReadModel(Chef)
      .compare((receivedState, currentState, embeddingContext) => {
          return receivedState.dishes
              .filter((dish: string) => !currentState.dishes.includes(dish))
              .map((missingDish: string) => new DishPrepared(missingDish, embeddingContext.key.value));
      })
      .deleteMethod((currentState, embeddingContext) => {
          return new ChefFired(currentState.name);
      })
      .on(DishPrepared, _ => _.keyFromProperty('Chef'), (chef, event, context) => {
          chef.name = event.Chef;
          if (!chef.dishes.includes(event.Dish)) chef.dishes.push(event.Dish);
          return chef;
      })
      .on(ChefFired, _ => _.keyFromProperty('Chef'), (chef, event, context) => {
          return ProjectionResult.delete;
      });
  })

• Syntax for an embedding class:

  
  import { compare, deleteMethod, embedding, EmbeddingContext, EmbeddingProjectContext, on } from '@dolittle/sdk.embeddings';
  import { ProjectionResult } from '@dolittle/sdk.projections';
  import { DishPrepared } from './DishPrepared';
  import { DishRemoved } from './DishRemoved';

@embedding('98f9db66-b6ca-4e5f-9fc3-638626c9ecfa') export class DishCounter { numberOfTimesPrepared: number = 0;

@compare()
compare(receivedState: DishCounter, embeddingContext: EmbeddingContext) {
    if (receivedState.numberOfTimesPrepared > this.numberOfTimesPrepared) {
        return new DishPrepared(embeddingContext.key.value, embeddingContext.key.value);
    }
}

@deleteMethod()
remove(embeddingContext: EmbeddingContext) {
    return new DishRemoved(embeddingContext.key.value);
}

@on(DishPrepared, _ => _.keyFromProperty('Dish'))
onDishPrepared(event: DishPrepared, context: EmbeddingProjectContext) {
    this.numberOfTimesPrepared ++;
}

@on(DishRemoved, _ => _.keyFromProperty('Dish'))
onDishRemoved(event: DishRemoved, context: EmbeddingProjectContext) {
    return ProjectionResult.delete;
}

}

- Syntax for getting embeddings:
```typescript
for (const [dish, { state: counter }] of await client.embeddings.forTenant(TenantId.development).getAll(DishCounter)) {
    console.log(`The kitchen has prepared ${dish} ${counter.numberOfTimesPrepared} times`);
}

const chef = await client.embeddings.forTenant(TenantId.development).get<Chef>(Chef, 'Mrs. Tex Mex');
console.log(`${chef.key} has prepared ${chef.state.dishes}`);

const dishes = await client.embeddings.forTenant(TenantId.development).getKeys(DishCounter);
console.log(`Got dem dish keys: ${dishes}`);