CertificateStoreIdentifier can't get private keys from Directories

[electro-logic]

Type of issue

• [X] Bug
• [ ] Enhancement
• [ ] Compliance
• [X] Question
• [ ] Help wanted

Current Behavior

When opening a Certificate Store and saving a certificate with a private key, the private key can't be read back. Have a look at this example

        static async Task Main(string[] args)
        {
            Directory.Delete("Certificates", true);
            X509Certificate2 cert = new X509Certificate2("cert.pfx", (string)null, X509KeyStorageFlags.Exportable);
            var certificateStoreIdentifier = new CertificateStoreIdentifier("Certificates\\My", CertificateStoreType.Directory, false);
            using (var store = certificateStoreIdentifier.OpenStore())
            {
                store.Open("Certificates\\My", false);
                Console.WriteLine(cert.HasPrivateKey);
                await store.Add(cert, null);
                store.Close();
            }
            using (var store = certificateStoreIdentifier.OpenStore())
            {
                store.Open("Certificates\\My", false);
                Console.WriteLine((await store.Enumerate()).First().HasPrivateKey);
                store.Close();
            }
        }

In real-world code the Find(true) function

ClientCertificate = await Configuration.SecurityConfiguration.ApplicationCertificate.Find(true).ConfigureAwait(false);

can't find a certificate because the private key is not found. Is this supposed to work this way?

Thanks

Expected Behavior

Private Key is returned when the store is read again

Steps To Reproduce

No response

Environment

- OS: Windows 11
- Environment: Visual Studio 2022
- Runtime: .NET 8
- Nuget Version: 1.5.374.126
- Component: Opc.Ua.Core
- Server: 
- Client:

Anything else?

Related issue: https://github.com/OPCFoundation/UA-.NETStandard/issues/2655

[mregen]

Hi @electro-logic, if the private key is to be used in the windows certificate store, it must be persisted. By default, the certificates created in this codebase try to use the ephemeral keystore to avoid that the private key is stored on disk. Once the cert is added to the windows certificate store it must be saved before as persisted and reloaded. There is code in the ua stack which implicitly handles these cases, the helper to use: X509Utils.CreateCopyWithPrivateKey. For your code sample it may work if you add the persist key flag: X509KeyStorageFlags.PersistKeySet

[electro-logic]

Hi @mregen,

The Enumerate() method (of the Opc.Ua.Core library) is not returning the private key when the certificate is stored in a directory. If you run the example you can see that True is printed (Private key present) but then False is printed when the certificate is read back. Adding the PersistKeySet flag is not fixing the issue.

This code is actually what is called when the Find(true) method is called on Configuration.SecurityConfiguration.ApplicationCertificate so other functions are not working properly when certificates are stored in directories.

Thanks

[salihgoncu]

Hi @electro-logic Is it possible to provide us the certificate in question? Or can we produce such a certificate consistently?

Best Regards

[electro-logic]

Hi @salihgoncu

The certificate is created (with the private key) in the example with the instruction

X509Certificate2 cert = new X509Certificate2("cert.pfx", (string)null, X509KeyStorageFlags.Exportable);

the certificate is then saved in the DirectoryStore. Two files are created (one is the .der public key, the other is the .pfx containing the public+private keys).

At this point because of a bug, the DirectoryCertificateStore.Enumerate() function can't find back the private key. This issue is preventing a proper use of certificates in directories.

[salihgoncu]

Hi @electro-logic,

I wasn't clear with my question I think. This cert.pfx file that is used to instantiate the new instance of X509Certificate2 object is my concern. - How was this file generated? Can we have a look at the parameters involved in the creation process? I tried different certificate files and they worked as expected. So, I suspect some parameters of the certificate file is differing and causing the private key not being persisted.

Best Regards

[electro-logic]

Hi @salihgoncu ,

I used the Opc.Ua.CertificateFactory.CreateCertificate() function (as below) to generate the certificate.

using Opc.Ua;
using System.Security.Cryptography.X509Certificates;
internal class Program
{
    static async Task Main(string[] args)
    {
        if (Directory.Exists("Certificates"))
            Directory.Delete("Certificates", true);

        var pfxCertBytes = CertificateFactory.CreateCertificate("urn:sample.client", "sample", null, new[] { "127.0.0.1" })
            .SetNotBefore(System.DateTime.Now)
            .SetNotAfter(System.DateTime.Now.AddMonths(24))
            .SetHashAlgorithm(X509Utils.GetRSAHashAlgorithmName(256))
            .SetRSAKeySize(2048)
            .CreateForRSA()
            .Export(X509ContentType.Pfx);

        X509Certificate2 cert = new X509Certificate2(pfxCertBytes, (string)null, X509KeyStorageFlags.PersistKeySet | X509KeyStorageFlags.Exportable);
        var csi = new CertificateStoreIdentifier("Certificates\\My", CertificateStoreType.Directory, false);
        using (DirectoryCertificateStore store = (DirectoryCertificateStore)csi.OpenStore())
        {
            store.Open("Certificates\\My", false);
            Console.WriteLine(cert.HasPrivateKey);
            await store.Add(cert, null);
            store.Close();
        }
        using (DirectoryCertificateStore store = (DirectoryCertificateStore)csi.OpenStore())
        {
            store.Open("Certificates\\My", false);
            Console.WriteLine((await store.Enumerate()).First().HasPrivateKey);
            store.Close();
        }
    }
}

[salihgoncu]

Hi @electro-logic,

Thanks a lot for the snippet. - I'll try to repro the issue. If I cannot repro, is it possible for us to have a call?

Best Regards

[electro-logic]

Hi @salihgoncu

Attached a mini project to reproduce the issue.

Issue.zip

I'm available to schedule a call if you can't reproduce the issue on your side.

Thanks

[mregen]

Hi @electro-logic, follow this code snippet to create a new cert and to reload the private key: https://github.com/OPCFoundation/UA-.NETStandard/blob/16b9aeff4d23efc01f7a80cfbd8ae0004d7dad3b/Libraries/Opc.Ua.Configuration/ApplicationInstance.cs#L797

Just enumerating the certs with Find(true) is not enough to load a private key from directory store in the windows key store, LoadPrivateKey must be used.

[electro-logic]

Hello @mregen

Using LoadPrivateKey() works, but is not very clean

        using (DirectoryCertificateStore store = (DirectoryCertificateStore)csi.OpenStore())
        {
            store.Open("Certificates\\My", false);
            var c1 = (await store.Enumerate()).First();
            c1 = await store.LoadPrivateKey(c1.Thumbprint, null, null);
            Console.WriteLine(c1.HasPrivateKey);
            store.Close();
        }

Would be much cleaner to have an Enumerate() method with a "LoadPrivateKeys" parameter.

1) What's the purpose of the "noPrivateKeys" parameter of the DirectoryCertificateStore.Open() method? 2) What's the difference between Configuration.SecurityConfiguration.ApplicationCertificate.Find(true) and Find(false)?

Thanks

[mregen]

Hi @electro-logic, that codebase has evolved over the years, has become a little cumbersome to use and cert management has some caveats on various platforms, so using the offered functions is the best way to ensure the code executes everywhere.

For question 1), there are cert stores which store public and private keys and some which store only public keys. To avoid that private keys are accidently written to storage, it can be opened with the flag no PrivateKeys.

2.) Find(true) is supposed to load the private key, but it works only for the X509Store store. Keys in the file system may be protected by a password, so LoadPrivateKey can provide that information. Better were to leave the Find(true) private as the outcome may be unpredictable (e.g. the private key is loaded but unusable because it is not in the right key store on windows.)

[electro-logic]

Thank you for the clarifications