Closed go-aegian closed 3 years ago
Can you post your (sanitized) HOCON configuration? and are you using the new setup class?
Basically the hocon remained the same when it's local so in this case is when it fails, just upgraded the package and run it locally and it failed.
tester {
class = "Akka.Persistence.Azure.Snapshot.AzureBlobSnapshotStore, Akka.Persistence.Azure"
# connection string, as described here: https://docs.microsoft.com/en-us/azure/storage/common/storage-configure-connection-string
connection-string = "UseDevelopmentStorage=true"
# the name of the Windows Azure Blob Store container used to persist snapshots
container-name = "default"
# Initial timeout to use when connecting to Azure Storage for the first time.
connect-timeout = 3s
# Timeouts for individual read, write, and delete requests to Azure Storage.
request-timeout = 3s
# Toggle for verbose logging. Logs individual requests to Azure.
# Intended only for debugging purposes. akka.loglevel must be set to DEBUG
# in order for this setting to be effective.
verbose-logging = off
# dispatcher used to drive snapshot storage actor
plugin-dispatcher = "akka.actor.default-dispatcher"
}
Can you rename tester
to akka.persistence.snapshot-store.azure-blob-store
and see if it works?
You also need to set akka.persistence.snapshot-store.plugin = "akka.persistence.snapshot-store.azure-blob-store"
in your configuration.
I may have found the problem, we will try to do a patch/small release and see if it fixes your problem.
great let me know I'll update the package and test it
https://github.com/petabridge/Akka.Persistence.Azure/releases/tag/0.8.1 - now live on NuGet @sainzg
@sainzg please re-open this issue if the problem persists
I have another class that deals straights to blob storage and after securing it, is able to read/write blobs
but when it comes to akka, I think I'm making a mistake somewhere as it seems the actor fails to connect to the container when restoring its state from snapshotoffer messages, or for some reason it does not use the container-name specified in hocon?
snapshot-store { plugin = "akka.persistence.snapshot-store.azure-blob-store"
actor-blob {
# qualified type name of the Azure Blob Storage persistence snapshot storage actor
class = "Akka.Persistence.Azure.Snapshot.AzureBlobSnapshotStore, Akka.Persistence.Azure"
# connection string, as described here: https://docs.microsoft.com/en-us/azure/storage/common/storage-configure-connection-string
connection-string = ""
# the name of the Windows Azure Blob Store container used to persist snapshots
container-name = "actorblobcontainer"
# Initial timeout to use when connecting to Azure Storage for the first time.
connect-timeout = 3s
# Timeouts for individual read, write, and delete requests to Azure Storage.
request-timeout = 3s
# Toggle for verbose logging. Logs individual requests to Azure.
# Intended only for debugging purposes. akka.loglevel must be set to DEBUG
# in order for this setting to be effective.
verbose-logging = off
# dispatcher used to drive snapshot storage actor
plugin-dispatcher = "akka.actor.default-dispatcher"
}
}
public static void Main(string[] args) {
actorSystem = ActorSystem.Create(ClusterHelper.ActorSystem, hoconConfig);
actorSystem.Settings.Setup.And(AzureBlobSnapshotSetup.Create(new Uri(blobConnection), new DefaultAzureCredential()));
var initialActor = actorSystem.ActorOf(InitialActor.Props(), nameof(InitialActor));
}
public class InitialActor : ReceivePersistentActor {
public InitialActor() {
SnapshotPluginId = "akka.persistence.snapshot-store.actor-blob";
Recover<SnapshotOffer>(HandleEvent);
}
private void HandleEvent(SnapshotOffer evt) {
if (evt.Snapshot is somestateclass) ....
}
}
Just to give you more insight, even when the blob storage is not locked down to private mode, and I basically don't even set this line actorSystem.Settings.Setup.And(AzureBlobSnapshotSetup.Create(new Uri(blobConnection), new DefaultAzureCredential())); The current package fails to restore snapshots so for the time being I'm rolling it back to 0.7.2 which works.
Hope this help and please let me know when you have another release and I'll test it out again and provide you with feedback
Can you change the way your ActorSystem
is initialized? I think its supposed to be:
var bootstrap = BootstrapSetup.Create()
.WithConfig(hoconConfig)
.And(AzureBlobSnapshotSetup.Create(new Uri(blobConnection), new DefaultAzureCredential()));
actorSystem = ActorSystem.Create(ClusterHelper.ActorSystem, bootstrap);
var initialActor = actorSystem.ActorOf(InitialActor.Props(), nameof(InitialActor));
@Arkatufus
Ok I did set it up that way you recommended and kept the version as 0.8.1.
Here are my findings
- Local storage works, as it uses the local connection and does not add the AzureBlobSnapshotSetup class to it.
Created new actor and saved its state, then updated state all fine.
- Cloud storage does not work.
Configured to be open as always with Blobs set as Public access level set to Container and Blob Storage open to public read.
This is for a brand new actor which does not have anything in the journal and/or snapshot blobs.
Error: Persistence failure when replaying events for persistenceId [Test-e12556ad-81cb-4a3c-9339-c988588c64c5]. Last known sequence number [0]2021-04-22T17:25:09.330528449ZAkka.Persistence.RecoveryTimedOutException: Recovery timed out, didn't get snapshot within 30s.
Are there any other Azure related errors before this happens, does the persistence plugin manage to connect correctly to the service? Have you set all of the credentials correctly?
Have you set up at least one of the proper way of connecting to Azure service using DefaultAzureCredential
?
The list of types of credential setup can be read here
Yes on the Azure side all is setup correctly because I ran another service outside Akka that reads/writes and uploads new blobs to another container, I can either secure the entire storage or just the container, connect using the same DefaultAzureCredential and all works just fine.
@sainzg I've looked further into this and found no problem with our code. Here is the code I've used to test that everything works: Code
Thank you so much. I will be able to test it again in a few weeks.
Getting this error when the connection is development and nothing is setup to run on azure
akka.persistence.snapshot-store.tester#1153123657]: Akka.Actor.ActorInitializationException: Exception during creation ---> System.AggregateException: One or more errors occurred. (Object reference not set to an instance of an object.) ---> System.NullReferenceException: Object reference not set to an instance of an object at Akka.Persistence.Azure.Snapshot.AzureBlobSnapshotStore.InitCloudStorage(Int32 remainingTries)
It might be this line needs to be using null interpolation in case nothing is setup