search

DevrexLabs / memstate

In-memory event-sourced ACID-transactional distributed object graph engine for .NET Standard
https://memstate.io
Other
363 stars 46 forks source link
csharp dotnet event-sourcing eventstore graph in-memory-database microservices origodb

readme

Memstate

Quickstart | Documentation | memstate.io/docs

Join the chat on Discord Build status NuGet Badge

What is Memstate?

In-memory event-sourced ACID-transactional replicated object graph engine. What? Can you say that again? Ok, it's an application server that keeps all your data in RAM. It runs without a database because all the transactions are recorded in a log and used to restore the state of the application at startup or when setting up a replica. You define the object model, commands and queries using C#. Besides being very simple Memstate is also very fast and will outperform any relational database.

Why Memstate?

Your data fits in RAM. Moving data back and forth between disk and memory is slow, difficult to maintain and error prone. Use Memstate to structure and manage your data in-memory, providing transparent persistence, high performance, high availability, concurrency control and transactions with strong ACID guarantees.

Memstate has many possible use cases but is designed primarily to handle complex OLTP workloads by replacing the datastore, data access layer and domain layer in a typical enterprise application.

The benefits are many:

So what's the catch?

Your data model needs to fit in RAM on a single server or VM. When using replication, each node has a copy of the entire model. This means Memstate can't scale beyond the limits of a VM with maximum amount of RAM. At the time of writing AWS offers 4TB instances which should be sufficient for most OLTP workloads.

As the journal grows over time, replaying billions of commands can take a significant amount of time which means restoring a node or setting up a new replica could take several hours.

Did we mention how simple Memstate is to use?

        [Test]
        public async Task Most_compact_start_using_all_default_configurations()
        {
            var engine = await Engine.Start<LoyaltyDB>();
            Print(await engine.Execute(new InitCustomer(10, 10)));
            Print(await engine.Execute(new InitCustomer(20, 20)));
            Print(await engine.Execute(new TransferPoints(10, 20, 5)));
            await engine.DisposeAsync();

            // Produces the following output :)

            /*             
            Customer[10] balance 10 points.
            Customer[20] balance 20 points.
            Sent 5 points. | Sender, Customer[10] balance 5 points. | Recipient, Customer[20] balance 25 points.
            */
        }

        private void Print(object o)
        {
            Console.WriteLine(o.ToString());
        }

Quickstart - getting started

Quickstart

Governance, Support and Contributions

Memstate is an open source project sponsored and governed by Devrex Labs, an LLC based in Sweden. Devrex Labs provide commercial support and consulting services for OrigoDB and Memstate. Contributions are welcome, check out the issues (or add a new one) and discuss with the core team before getting started. You will need to sign our CLA using the CLA assistant

Background and Objectives

Devrex Labs also maintain OrigoDB, an in-memory database engine for .NET Framework. Memstate is a redesign based on our experience building and working with OrigoDB, taking the best parts, learning from mistakes and setting some new objectives. This section takes on a comparison with OrigoDB perspective.

Memstate's replication scheme has higher availability (because any node can accept writes) at the expense of some temporal inconsistency across the nodes.

Features

In-memory object graph

Your data fits in RAM. Define your own object model and operations (commands and queries) or use a built-in model such as redis, keyvalue, graph, relational, document, sparse array, etc. Also we have some cool geo-spatial types which you can use in your custom model.

Event-sourced

All operations (commands) are written to persistent storage and used to restore the state of the in-memory object graph when a node starts up. The log is a complete audit trail of every single change to the system and can be used to recreate the state of the graph at any given point in time. A point-in-time snapshot can be used for queries or historical debugging.

Real-time event notifications

Domain events emitted from within the model can be subscribed to (with optional server side filtering) by local or remote clients. This enables reactive applicatons with real time updates when data is changed.

Configurable serialization formats

Choose JSON for readability and interoperability or a binary format for speed and smaller size.

ACID Transactions

Configurable Storage Backend

Memstate relies on a backing storage provider for persistence and global message ordering. We currently support EventStore, PostgreSQL and plain old file system storage.

Memstate in the press