Cluster On Premise

[AmirSasson]

Hi, We are considering the use of the orleans framework as an alternative to SQL transactions. when I run 2 silos on my machine and running a load of transactions, then suddenly kill the primary Silo, seems like there is a lose of information, and the secondary silo doesnt take over to handle the queued transactions of the grains. Since i am talking about financial transaction, i need it to be fully trust application. can the orleans framework support this? should i implement an external queue provider?

my use cases are:

1. Deposit to User wallet
2. withdraw from User Wallet
3. Query the User Wallet balance

Any help is much appreciated.

[veikkoeeva]

@AmirSasson If you are using MembershipTableGrain, the deployment won't survive a primary silo crash. See documented at Server Configuration:

For development or if it’s not possible to use Azure Table, silos can be configured to use the membership grain instead. Such a configuration is unreliable as it will not survive a failure of the primary silo that hosts the membership grain. “MembershipTableGrain” is the default value of LivenessType.

The documents mention in that paragraph only Azure Table, but for a reliable deployment you can also use ZooKeeper or SQL Server (or some other relational backend). For me at least the SQL Server topology on LocalDb feels quite performant. The documents are going through a bit more churn as of now, I suppose, as there has been quite a lot development on various parts.

Note, this is only about reliable deployment.

<edit: A relevant part of configuration on SQL Server is

<Globals>
<Liveness LivenessType ="SqlServer" />
<SystemStore SystemStoreType ="SqlServer"
             DeploymentId="JustAnExample"
             DataConnectionString="Data Source=(localdb)\MSSQLLocalDB;Database=Orleans;Integrated Security=True;Asynchronous Processing=True;Max Pool Size=200; MultipleActiveResultSets=True" />

Where Orleans is a pre-created database with script CreateOrleansTables_SqlServer.sql.

[AmirSasson]

Thank a lot Veikko, i tried it, i have another issue that relates. when i run a load test, seems like the transactions are kept in memory queue, and when i shut down the primary server, the transaction are gone and not migrated to the secondary. is that correct? what do i need to do in order to maintain the queued transactions and that the secondary will take over and handle them?

[veikkoeeva]

@AmirSasson Are you using Orleans.Providers.Streams.SimpleMessageStream.SimpleMessageStreamProvider and Orleans.Storage.MemoryStorage? As it happens, currently there is no stream provider for relational storage (e.g. SQL Server).

There's a long running relational storage thread at https://github.com/dotnet/orleans/issues/255. Some of the things have been implemented already. I think I would like to have stream on top relational database within some months. I have some ideas on the relational part already, which might get to the ballpark of 5 k–100 k messages/second (yes, varies wildly). Likely I would try something like a preallocated table mimicking a ring buffer (e.g. inserting to the next slot/row), which shouldn't be much slower than adding rows to a preallocated and memory-mapped filesystem file. Naturally nothing has been done as of yet in that regard. It feels as if when I start it shouldn't take that long, albeit anyone else is free to do it also. Hmm, maybe I should record an issue for that already.

<edit: Maybe something like http://rusanu.com/2010/03/26/using-tables-as-queues/, but SQL Server 2012 and newer have SEQUENCE ids, which likely can be used to make faster inserts.

[gabikliot]

@AmirSasson , Orleans does not directly implement transactions or durable queues. Orleans is used for building an in-memory middle-tier "business logic". The grains (Orleans's main computational abstraction) are kept in memory and their in memory state is thus lost when server (silo) crashes.

In order to build a higher level transactions and durable guarantees there are generally 2 approaches - "after" and "before" the grain. Both approaches externalize the transactions or durability to some outside component but allows easy integration of that external component with the rest of Orleans.

1) Use grain persistence. Orleans allows to store grain state in a configured durable store. http://dotnet.github.io/orleans/Getting-Started-With-Orleans/Developing-a-Grain -> "Persistence of Grain State" http://dotnet.github.io/orleans/Step-by-step-Tutorials/Declarative-Persistence

There are already a number of Persistence Providers: Azure Table, Blobd, Redis, DocumentDB, SQL, https://github.com/OrleansContrib

In if you are using grain persistence, you will store your transaction in grain's state and call WriteAsync every time the state changes. That way all your changes will be stored durably.

2) Use Persistent Stream Provider - in this approach you will store all your transactions in a durable external queue and Orleans will deliver them from the queue to the grain. http://dotnet.github.io/orleans/Orleans-Streams/Stream-Providers

[gabikliot]

@veikkoeeva, as for your idea about "Using tables as Queues" I think it is quite an interesting idea in general. Not immediately applicable to Orleans , but might be in the future. I wrote more it about here: https://gitter.im/dotnet/orleans?at=55ac262f43d782015a9e801a

[veikkoeeva]

@gabikliot I'll provide the correct link here too: Databases at Box: When scale, collaboration and enterprise users collide.

I'm thinking of using table as a basis for a stream. I believe it is a rather common use case scenario in an enterprise setting. Unfortunately a bit tricky to pull off on relational database, but I think one could get decently close to disk speed with writes. Reads are likely memory bound. Take this with a pinch of salt, I'm not what one might call an expert, but I think I can pull this off. :)

[gabikliot]

@veikkoeeva , do you mind opening a separate issue about that? I think it is too interesting and important to be hidden within another unrelated issue. I will also post more thoughts there.

[veikkoeeva]

@gabikliot Will do tomorrow. Now I need to stretch myself to bed before the keyboard becomes my pillow.

[gabikliot]

@AmirSasson, do you still need help on that issue? Can we close it?

[AmirSasson]

Hi mate, Can be closed. We found some Israeli guy named “Israel Olcha” that might help us with our challenge on this POC. I am keeping your Skype address, if I reach any dead end. Thank you so much Gabi.

Btw, Do you know about any financial transaction system that uses the Orleans as a wallet manager?

tx

[gabikliot]

I am glad you guys are making progress. Don't hesitate to address us if you get stuck.

Re financial transaction system that uses the Orleans: you may want to talk to @galvesribeiro. He just started working on something related to financial transaction over Orleans.