search

MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.2k stars 21.35k forks source link

"Invoke-WebRequest : Unable to retrieve certificates because the thumbprint is not valid. Verify the thumbprint and retry" #20918

Closed starforce closed 5 years ago

starforce commented 5 years ago

Hello, I have a question about using the Invoke-WebRequest power shell command in your example.

First let me give you some background information about my current setup. Right now, I have a Service Fabric cluster using an X.509 cert from an CA and is used to secure the cluster. I have Admin rights, Also, I have the certificate on my local computer in the My (Personal) store. I set up AAD when I first created the cluster which allows me to use Service Fabric Explorer and publish/update code from Visual Studio 2017. This all works fine ,but The problem I am having is when I try to use the Invoke-WebRequest in your example and specify the -CertificateThumbprint. I am using the same Thumbprint that I use to set up my Cluster that I got from the CA. However, I always get the following error

"Invoke-WebRequest : Unable to retrieve certificates because the thumbprint is not valid. Verify the thumbprint and retry"

Is there some reason why I cannot use the same X.509 Thumbprint and Cert that I use for publishing code from Visual Studio to My service fabric cluster and for Service Fabric Explorer ? According to Microsoft documentation, "By default the cluster certificate has admin client privileges." So confused as to why this isn't working.

Thank you

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

starforce commented 5 years ago

I just got my DigiCert. Will continue on Monday..

starforce commented 5 years ago

@aljo-microsoft

I plan on creating my Test Service Fabric Cluster and Traffic Manager through portal.zure.com instead of trying to modify this very long script.

https://github.com/aljo-microsoft/demo/blob/master/deploy/Deploy-2NodeTypes-3ScaleSets.endpoint.json

starforce commented 5 years ago

@hrushib @jakaruna-MSFT @MicahMcKittrick-MSFT

Hello @aljo-microsoft

Are you saying that if I run this script that you provided here for creating a service fabric cluster

https://github.com/aljo-microsoft/demo/blob/master/deploy/Deploy-2NodeTypes-3ScaleSets.endpoint.json

Instead of creating my cluster through the Azure Portal that I can set the cluster name to my CN in my cert I got DigiCert? Because when I do this from the portal, there is no way to get around creating a cluster without having a Microsoft domain name. See below. I just want to make sure I can before creating the cluster manually. Because according to you “This will allow you to provision your cluster using common name and a 3rd party custom domain “ See below.

image

starforce commented 5 years ago

@hrushib @jakaruna-MSFT @MicahMcKittrick-MSFT @aljo-microsoft

Hey thanks for all the help but none of the suggestions fixed my problems. Going back to my original issue:

Invoke-WebRequest : Unable to retrieve certificates because the thumbprint is not valid. Verify the thumbprint and retry" Is there some reason why I cannot use the same X.509 Thumbprint and Cert that I use for publishing code from Visual Studio to My service fabric cluster and for Service Fabric Explorer ? According to Microsoft documentation, "By default the cluster certificate has admin client privileges." So confused as to why this isn't working.

This really should work. It works currently for deploying code to Service Fabric inside of Visual Studio and it works with Service Fabric explorer when there is a custom Domain. Need to get some help from an Microsoft Engineer but I spent a lot of time on this and I must move on. Also, These documents need to reflect that some use powershell scripts to perform their work but others just use portal.azure.com or resources.azure.com because it is easier and safer. So many of the examples here assume that powershell scripts are use. In a big organization with a SCM department this can be true but for smaller companies, we don’t’ have the luxury of fulltime SCM. Now I understand for backup there isn’t a UI but, from my point of view, while there is no doubt that I can use and run PowerShell scripts, it is not the best way because it is easier to make errors than using a UI.

aljo-microsoft commented 5 years ago

@starforce Portal isn't appropriate for your use case, and you should use an ARM template; you can start by modifying the template I provided, or you can buy Premier Engineering support if you need help authoring your use case specific JSON file using Rest API docs, or content I provided.

resources.azure.com and portal are not safer or recommended for production Service Fabric environments; you simply are not getting the full benefits of ARM using either of them, and will not always be able to leverage the latest Azure resource features through either of them.

So this isn't the original issue, as the only SF specific thing your initial issue identified that I've since documented and filled, is how to update the management URL for Service Fabric Clusters in portal for your custom domain (with you being advised to use an ARM template).

Azure provides tutorials for how to host your custom domain in Azure: https://docs.microsoft.com/azure/dns/dns-delegate-domain-azure-dns

Hosting your domain in Azure is a not a feature of SF resource's, as you can run your Azure hosted service on many resources; SF, App Services, AKS, ect.

I've also not only provisioned a 3rd Party Cert, but completed end to end configuration of hosting my service in Azure using that cert from Digit cert; which they issued me without requiring I provision any Azure resources first to host some string, and my cluster using that cert is still running today and resolves my custom domain to my environment.

My recommendation given the struggles you are having is to buy appropriate support to provide the velocity of Hosting your domain in Azure that you desire.

starforce commented 5 years ago

Thank you for all your help, and I know how to use DNS hosting and all that technology because I got the same technology with Google domains. Also, I know how to modify arm templates . But the issue is there is no documentation that says that you can get around using a Microsoft domain when creating a service fabric cluster by using an arm template. And the script has over 1600 lines which is error-prone for human beings . I think a better solution is for me to call my internal apis at night and just retrieve the data in a Json format and save it myself until this featured becomes more robust and user friendly. And if you cannot use the portal to create the service fabric explore clusters in production then that needs to be explicitly stated in the how to documentation. If you do a search on Google of other people's examples they always use the portal to create service fabric clusters even for custom domains. My production website has been running for the last 6 months and I didn't use one arm template to create it and I have SSL working.

aljo-microsoft commented 5 years ago

@starforce

I've published public docs that say clearly you should use ARM to provision production solutions, explicitly:

" In a production scenario, create Azure Service Fabric clusters using Resource Manager templates. Resource Manager templates provide greater control of resource properties and ensure that you have a consistent resource model. "

https://docs.microsoft.com/azure/service-fabric/service-fabric-best-practices-infrastructure-as-code

I'm glad your issue is resolved, and highly recommend reviewing all of our best practice documentation to avoid future issues:

https://docs.microsoft.com/azure/service-fabric/service-fabric-best-practices-overview

starforce commented 5 years ago

Yes you also said that I could get a certificate for a Microsoft domain which I told you you could not. Thank you and have a good one

aljo-microsoft commented 5 years ago

Only authorized MSFT's like myself can request certificates for Microsoft owned domains.

starforce commented 5 years ago

Yes but you should have known this before you gave me that solution that didn't work for me. I lost two full days working on that solution that you provided me. And now you ask me to modify a 1600 Plus line script and lose another two days on something that might work. Clearly Microsoft doesn't understand all the work and effort it takes to get things running. Anytime I can use a front-end UI to simplify my life I do.

@MicahMcKittrick-MSFT is this normal?

aljo-microsoft commented 5 years ago

@starforce

Please Review the following Best Practice Documentation: https://docs.microsoft.com/azure/service-fabric/service-fabric-best-practices-overview https://docs.microsoft.com/azure/service-fabric/service-fabric-best-practices-security https://docs.microsoft.com/azure/service-fabric/service-fabric-best-practices-networking https://docs.microsoft.com/azure/service-fabric/service-fabric-production-readiness-checklist

Also given my understanding of your issues being faced, in addition to a support ticket and above documentation recommendations, I recommend reviewing Keyvault and Networking resource documentation.

As always please open a support ticket to expedite resolution of your production Service Fabric solution.

The scenario as I understand it is: Service Fabric Client API accessibility from a users development environment PowerShell session, when passing their production 3rd party certificate common name, to the Invoke-WebRequest PS Function.

Issue kind: Security and Networking

Issue Description:
3rd party domain naming resolution for a users deployed Azure Service Fabric Cluster Management API with SSL enabled.

Current State: Unable to reproduce user issue.

01/29/2019 and previous dates performed validation of: Key Vault Integrated Digit Cert CA issued certificate, for 3rd party registered (GoDaddy) domain, used in successful execution of Invoke-WebRequest to a Secure Azure Service Fabric Cluster.

Validating Azure Service Fabric Clusters best practices for securing your clusters, is declaring your Certificate Authority issued certificate properties Common Name and Issuer Thumbprint in your Azure Resource Manager template.

Ref: https://docs.microsoft.com/azure/service-fabric/service-fabric-best-practices-infrastructure-as-code https://docs.microsoft.com/azure/service-fabric/service-fabric-best-practices-security