jet / dotnet-templates

Example app and service templates `dotnet new -i Equinox.Templates; dotnet new eqx*/pro*` https://github.com/jet/equinox https://github.com/jet/FsCodec
https://github.com/jet/propulsion
Apache License 2.0
64 stars 16 forks source link

Add EventStore source support to `sync` Template #16

Closed bartelink closed 5 years ago

bartelink commented 5 years ago

~While this works, provides good throughput and other things, there's still stuff to do so it's really not ready to "review" in the sense of evaluating "is this a clean simple ingester?" atm (clue: atm the code is too messy, and there is an obvious need to add support for doing multiple concurrent readers)~

~TL;DR at the moment its a POC EventStore->Equinox.CosmosDb one-way ingester, but the general principle is that there are lots of uses for an efficient bulk sync tool working at the stream level in order to optimize ingesting performance (and batch writes to same stream) - this allows one to subset by Category and/or do transforms when propagating data onward~

~This is approaching the final shape of things in that - the Sync program is intended to be a continuous app you run in prod that can support Cosmos -> Cosmos and EventStore -> Cosmos ingestion - the Ingest Program only applies to EventStore -> Cosmos loading and is intended to be run once on a standalone basis~

The equinox-sync template's Sync app includes support for both the striped reading (which was originally pioneered in an Ingest application) (badged gorge mode ,-g) and working as a more traditional projector (which it reverts to after any gorge mode)

Work list:

bartelink commented 5 years ago

Marking as ready for review - I still have some tidying to do, but the basic split between Sync's EventStore mode and the Ingest Program (which started life as a Template) is now in place.