Moving vault from one pc to another.

[centur]

Hi, I tried to move my LE vault with all certs to a VM (so I can share my workflow with other team members. I've backed up it with this script (this is a remake of something I found in other discussions, (c) of the original author)

Add-Type -assembly "System.IO.Compression.FileSystem";
$suffix = Get-Date -Format "yyyy-MM-dd"
$sourceFolder = (Get-ACMEVaultProfile).VaultParameters['RootPath']
$targetFile = "$PSScriptRoot\\ACMESharpVault-$suffix.zip" 
[IO.Compression.ZipFile]::CreateFromDirectory( $sourceFolder, $targetFile)
Write-Output "ZipFile Created : $targetFile"

and restored it with

Add-Type -assembly "System.IO.Compression.FileSystem";
$suffix = Get-Date -Format "yyyy-MM-dd"
$sourceFile = "$PSScriptRoot\\ACMESharpVault-$suffix.zip" 
$targetFolder = (Get-ACMEVaultProfile).VaultParameters['RootPath']
[IO.Compression.ZipFile]::ExtractToDirectory($sourceFile, $targetFolder)
Write-Output "ZipFile restored to : $targetFolder"

When I tried to renew cert with my other commandlets - I got this :

Submit-ACMECertificate Error creating new cert Authorizations for these names not found or expired: drawboard.com

For some reason - I ran another vault registration New-ACMERegistration -Contacts "$contacts" -AcceptTos

Now I have 2 registrations and broken Vault :(

Also worth to note that my machine vault RootPath is pointing to user\local folder while that VM one RootPath is C:\ProgramData\ACMESharp\sysVault

Any idea what can be broken in my procedure ?

[ebekker]

Hmm, the second registration should not have clobbered the first (in theory) -- there are checks in place to prevent that, will need to check on that...

Fortunately, it sounds like you're in a good position since you have a backup in the form of your ZIP file, so you can just delete/move the clobbered vault folder, and restore your original ZIP file.

Now, to address your first problem (Submit-ACMECertificate error), were you able to complete the preceding step successfully, namely the New-ACMECertificate call?

Can you list out the certificates there with a Get-ACMECertificate and see that the one(s) you expect is there?

[centur]

I'll redeploy the vault a bit later, from what was there - yes, I can list all the necessary certs with no issues - i have some posh-ui scripts to do it nicely. Will check and come with exact errors from the fresh vault

[centur]

Ok, managed to get this run from scratch - cleaned up vault, restored it from zip again, run my script step by step. Here is an output:

Posh> Get-ACMEIdentifier |FT

Seq Id                                   Alias                                Label Dns                         Status
--- --                                   -----                                ----- ---                         ------
  0 b02fbc6e-a375-4c76-b31c-0943c5e972cf alias-drawboard.com                        drawboard.com               valid

Posh> $certToRenew = Get-ACMEIdentifier | ?{$_.Status -eq 'valid'} | Out-GridView -PassThru -Title "Select VALID Certificate to renew"
Posh> $certToRenew

Seq    : 0
Id     : b02fbc6e-a375-4c76-b31c-0943c5e972cf
Alias  : alias-drawboard.com
Label  :
Dns    : drawboard.com
Status : valid

Posh>
Posh> $dnsName = $certToRenew.Dns
Posh> $dnsAlias = $certToRenew.Alias
Posh>
Posh> # Get correct top-domain to include
Posh> $parts = $dnsAlias -split "\.";
Posh> $topLevelDomain = $parts[-2..-1] -join "."
Posh> $includeDomain = Get-ACMEIdentifier  | ?{$_.Dns -eq $topLevelDomain} | ?{$_.Status -eq 'valid'}
Posh> $dnsExtraDomains = @($includeDomain.Alias, $dnsAlias)
Posh> Write-Host $dnsName
drawboard.com
Posh> Write-Host $dnsAlias
alias-drawboard.com
Posh>
Posh> $certAlias = "$dnsName $dateSuffix"
Posh> $pathToCertificates = "$CertRoot/$dnsName/$dateSuffix" # Where all certs should be imported to
Posh>
Posh> #Submit Certificate
Posh> Write-Host "===================================================================================="
====================================================================================
Posh> Write-Host "Submitting the certificate: ($dnsAlias) certAlias:($certAlias) ..."
Submitting the certificate: (alias-drawboard.com) certAlias:(drawboard.com ) ...
Posh> Write-Host $dnsExtraDomains
 alias-drawboard.com
Posh> Write-Host "===================================================================================="
====================================================================================
Posh>
Posh> New-ACMECertificate $dnsAlias -AlternativeIdentifierRefs $dnsExtraDomains -Alias $certAlias -Generate
New-ACMECertificate : invalid or missing reference
Parameter name: ref
At line:1 char:1
+ New-ACMECertificate $dnsAlias -AlternativeIdentifierRefs $dnsExtraDom ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [New-ACMECertificate], ArgumentNullException
    + FullyQualifiedErrorId : System.ArgumentNullException,ACMESharp.POSH.NewCertificate

Posh> $dnsExtraDomains
alias-drawboard.com
Posh>

So this happens when I'm trying to re-issue top level domain cert with included same top-level doman in -AlternativeIdentifierRefs ( which may be a stupid idea or another type of issue)

Another result is when I'm submitting subdomain with top level domain included in the certificate:

Renew Existing Cert and include parent domain
Existing valid certs in the vault:

Seq Id                                   Alias                             Label Dns                         Status
--- --                                   -----                             ----- ---                         ------
  0 b02fbc6e-a375-4c76-b31c-0943c5e972cf alias-drawboard.com                     drawboard.com               valid
 ---
 24 67747687-950c-43ab-b777-14d9d8fc83bd alias-legacy-uat.drawboard.com          legacy-uat.drawboard.com    valid

Selected Certificate is :
{ Seq = 24, Id = 67747687-950c-43ab-b777-14d9d8fc83bd, Alias = alias-legacy-uat.drawboard.com, Label = , Dns = legacy-uat.drawboard.com, Status = valid }
====================================================================================
===================== Starting renewal procedure ===================================
====================================================================================
legacy-uat.drawboard.com
alias-legacy-uat.drawboard.com
====================================================================================
Submitting the certificate: (alias-legacy-uat.drawboard.com) certAlias:(legacy-uat.drawboard.com 2017-Jan-18_23-40) ...
alias-drawboard.com alias-legacy-uat.drawboard.com
====================================================================================

Id                       : 30d9ca1f-d298-438c-b4bf-d64965a29f89
Alias                    : legacy-uat.drawboard.com 2017-Jan-18_23-40
Label                    :
Memo                     :
IdentifierRef            : 67747687-950c-43ab-b777-14d9d8fc83bd
IdentifierDns            : legacy-uat.drawboard.com
AlternativeIdentifierDns : {drawboard.com, legacy-uat.drawboard.com}
KeyPemFile               :
CsrPemFile               :
GenerateDetailsFile      : 30d9ca1f-d298-438c-b4bf-d64965a29f89-gen.json
CertificateRequest       :
CrtPemFile               :
CrtDerFile               :
IssuerSerialNumber       :
SerialNumber             :
Thumbprint               :
Signature                :
SignatureAlgorithm       :

Submit-ACMECertificate : Error creating new cert :: Authorizations for these names not found or expired: drawboard.com, legacy-uat.drawboard.com
At C:\Users\ops\Documents\LetsEncrypt\Renew-WithParent.LetsEncrypt.ps1:65 char:1
+ Submit-ACMECertificate "$certAlias"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : PermissionDenied: (ACMESharp.Vault.Model.CertificateInfo:CertificateInfo) [Submit-ACMECertificate], AcmeWebException
    + FullyQualifiedErrorId : urn:acme:error:unauthorized (403),ACMESharp.POSH.SubmitCertificate
refresh certificate details...

Update-ACMECertificate : Certificate has not been submitted yet; cannot update status
At C:\Users\ops\Documents\LetsEncrypt\Renew-WithParent.LetsEncrypt.ps1:69 char:1
+ Update-ACMECertificate $certAlias
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Update-ACMECertificate], Exception
    + FullyQualifiedErrorId : System.Exception,ACMESharp.POSH.UpdateCertificate

at this point script passes

New-ACMECertificate $dnsAlias -AlternativeIdentifierRefs $dnsExtraDomains -Alias $certAlias -Generate 

but fails on the next line

Submit-ACMECertificate "$certAlias"

and fails all consequent refreshes and checks.

Interesting message is this:

Authorizations for these names not found or expired: drawboard.com, legacy-uat.drawboard.com

UPD: Re-run this cert re-issuing on my machine and got the same response. Seems that the problem is not with the vault =( . Checked all DNS TXT records for LE (_acme-challenge) and they are all there. Is there some kind of hidden expration on the domain challenge ? I've used same domain records for getting LE certs 3 times already and thought that they are valid forever

[ebekker]

Ah, I see the issue. You need to take a look at the most recent few comments in #165. You may also find the Reference Sheet useful.

[centur]

Great, thanks for the helpful hint.

If I groked it correctly - all dns domain name validation challenges are now expiring in 7 days, thus dns-01 challenge is not something that we can automate without domain records automation (as we need to re-generate domain records every 60 days :( if we want to roll out new certs every 60 days).

In terms of ACMESharp commands - I can't just call Complete-ACMEChallenge $dnsAlias -ChallengeType dns-01 -Handler manual once per domain, but must rerun it on almost every certificate re-issuing and handle manual dns-01 challenge. Is this correct ?

[centur]

Also this makes vault moving pretty redundant as there is nothing that one can reuse from existing vault, right ?
And it's simpler to start generating new certs every time than trying to renew the existing ones as they both would require new challenge to be completed. As well as there is no point to re-use the aliases in AcmeSharp as they are now disposable and there is no difference between creating a new alias or re-using any of previously generated because existing record no longer gives any advantage over new-ing one, like it was before (no challenge needed).

[ebekker]

Yes, you are correct, you need to re-authorize Identifiers on a recurring basis.

[ebekker]

The only reason to "reuse" a vault is if you want to keep all the related entities (Registration, Identifiers, Authorizations, Certificates) under one umbrella, but if you don't care about that -- you are correct, you can just re-generate a new vault each time and repeat registration.

If you're interested in the former, one thing that may interest you is a new addition in the upcoming 0.8.2 release. Right now there is only a single default file-based provider named local, but in the upcoming release there is going to be a new vault provider that will be based on a centralized storage service (akin to a NoSQL service) and you can just connect to that centralized vault from multiple hosts and achieve the same result as moving/copying the vault files around.

[centur]

Sounds interesting. That centralized vault will be in form of api service ? is it possible to self host it ?

[ebekker]

Right now the one I'm working on is backed by Firebase. The next one will be backed by EF Core, so you can host that one by using any supported SQL database provider

[centur]

What kind of interface you have for reading\saving settings ? Would it be possible to write a provider to store vault in S3 buckets or Azure Blobs\Table ? Hosting a SQL database for a vault which is 2MB in total is a bit overkill for small usage scenarios :(

[ebekker]

I agree -- SQL Server backend would be overkill, but by writing to EF Core, you can target any supported relational DB provider (SQLite) or even NoSQL DB in the future.

In any event the Firebase provider would be an example of a non-relational, document store. But you can implement your own based on this interface and use the default local vault provider as an example.