search

akkadotnet / Akka.Persistence.Redis

Redis storage for Akka.NET Persistence
Apache License 2.0
30 stars 19 forks source link

Akka.Persistence.Redis v1.4.16 - the "we removed Akka.Persistence.Query support" thread #126

Open Aaronontheweb opened 3 years ago

Aaronontheweb commented 3 years ago

In the course of investigating https://github.com/akkadotnet/Akka.Persistence.Redis/issues/109, we came to some conclusions about the use of Akka.Persistence.Query in Akka.Persitsence.Redis:

  1. It's not all that straightforward to implement in clustered Redis scenarios (including Twemproxy) - as individual entities need to be persisted into a single Redis node inside the cluster in order for routing to function correctly, and tags will need to be stored into a single node as well and there's a very good chance these two nodes will not be the same. Therefore, we can't guarantee that the IPersistentRepresentation and the Tagged referencing it can be written in the same atomic transaction. That's a bit of a problem.
  2. It's non-performant - the Akka.Persistence.Redis tests that leverage Akka.Persistence.Query took half an hour to run on a single node. Once we removed Akka.Persistence.Query functionality from the RedisJournal and removed the tests that covered it, the entire single node test suite ran in under 60 seconds. You can see some benchmark values here: https://github.com/akkadotnet/Akka.Persistence.Redis/pull/114#issuecomment-773706606
  3. Akka.Persistence.Query isn't a great fit for Redis - if you want flexible read / write queries, use a RDBMS or MongoDb. If you want blazing fast reads / writes for rapid-fire event sourcing, use this plugin.

We're opening this issue so we can get data from users this change adversely affected - if you were previously using Akka.Persistence.Query with Akka.Persistence.Redis and need that functionality back, please let us know and tell us a bit about your use case here!

andyfurnival commented 3 years ago

@Aaronontheweb I was trying to get your Cluster Sharding Viewer example working with Redis, as I wanted to understand why I'm getting occasional issues with an issue that was seen a long time, which causes the message "System.ArgumentException: Shard 98 is already allocated (Parameter 'e')" on Persistence Recovery.

I was hoping to interrogate Redis when the system is in a bad state, and try to deserialise the data and see what gets thrown up (and avoid all the boilerplate trying to go direct to redis to pull the raw data and try to deserialise it myself)

Given this notice above, am I right in thinking that any use of Persistent Query will simply not work for Redis? I'm not able to switch to ddata just yet, so its either Redis or I may need to look at an alternative plugin

Aaronontheweb commented 3 years ago

I was trying to get your Cluster Sharding Viewer example working with Redis

Which sample is this?

andyfurnival commented 3 years ago

https://github.com/Aaronontheweb/Cluster.Sharding.Viewer this one.

Aaronontheweb commented 3 years ago

Ah I see - good news though, this query:

https://github.com/Aaronontheweb/Cluster.Sharding.Viewer/blob/d19012d23ac1664608e8fbe27fddbebf5e549d14/src/Cluster.Sharding.Viewer/Program.cs#L22-L23

Is supported by Akka.Persistence.Redis.Query since it doesn't depend on tags. https://github.com/akkadotnet/Akka.Persistence.Redis/blob/4aa8cd9ce5a6a9039ca649bddcc2cde7049789f3/src/Akka.Persistence.Redis/Query/RedisReadJournal.cs#L103-L116

So does that viewer not run correctly with Redis right now?

andyfurnival commented 3 years ago

No, I can see query whilst monitoring redis which should return data (it does when called in redis-cli), but nothing ever returns in the console app. Might be serialisers. I’ll keep at it, now I know it should work