dotnet / aspnetcore

ASP.NET Core is a cross-platform .NET framework for building modern cloud-based web applications on Windows, Mac, or Linux.
https://asp.net
MIT License
34.59k stars 9.79k forks source link

Can't create development certificate on macOS Catalina #19590

Open rmarinho opened 4 years ago

rmarinho commented 4 years ago

Describe the bug

Trying to generate a development certificate on my macOS Catalina (10.15.4 Beta (19E242d)) using the dev-certs tool but is not not working.

output:

iRuiMSFT-MBP:~ rmarinho$ dotnet dev-certs https
A valid HTTPS certificate with a key accessible across security partitions was not found. The following command will run to fix it:
'sudo security set-key-partition-list -D localhost -S unsigned:,teamid:UBF8T346G9'
This command will make the certificate key accessible across security partitions and might prompt you for your password. For more information see: https://aka.ms/aspnetcore/2.1/troubleshootcertissues
A valid HTTPS certificate with a key accessible across security partitions was not found. The following command will run to fix it:
'sudo security set-key-partition-list -D localhost -S unsigned:,teamid:UBF8T346G9'
This command will make the certificate key accessible across security partitions and might prompt you for your password. For more information see: https://aka.ms/aspnetcore/3.1/troubleshootcertissues
password to unlock /Users/rmarinho/Library/Keychains/login.keychain-db: 
keychain: "/Users/rmarinho/Library/Keychains/login.keychain-db"
version: 512
class: 0x00000011 
attributes:
    0x00000000 <uint32>=<NULL>
    0x00000001 <blob>="com.apple.AppleMediaServices.mediaToken.macappstore"
    0x00000002 <blob>=<NULL>
    0x00000003 <uint32>=<NULL>
    0x00000004 <uint32>=<NULL>
    0x00000005 <uint32>=<NULL>
    0x00000006 <blob>=<NULL>
    0x00000007 <blob>=<NULL>
    0x00000008 <blob>=<NULL>
    0x00000009 <uint32>=0x00000000 
    0x0000000A <uint32>=0x00000000 
    0x0000000B <uint32>=0x00000000 
    0x0000000C <blob>=<NULL>
    0x0000000D <blob>=<NULL>
    0x0000000E <uint32>=<NULL>
    0x0000000F <uint32>=<NULL>
    0x00000010 <uint32>=<NULL>
    0x00000011 <uint32>=<NULL>
    0x00000012 <uint32>=<NULL>
    0x00000013 <uint32>=<NULL>
    0x00000014 <uint32>=<NULL>
    0x00000015 <uint32>=<NULL>
    0x00000016 <uint32>=<NULL>
    0x00000017 <uint32>=<NULL>
    0x00000018 <uint32>=<NULL>
    0x00000019 <uint32>=<NULL>
    0x0000001A <uint32>=<NULL>
security: SecKeychainItemCopyAccess: A missing value was detected.
Something went wrong. The HTTPS developer certificate could not be created.

To Reproduce

Running

dotnet dev-certs https

Further technical details

iRuiMSFT-MBP:~ rmarinho$ dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   3.1.102
 Commit:    573d158fea

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.15
 OS Platform: Darwin
 RID:         osx.10.15-x64
 Base Path:   /usr/local/share/dotnet/sdk/3.1.102/

Host (useful for support):
  Version: 3.1.2
  Commit:  916b5cba26

.NET Core SDKs installed:
  2.1.4 [/usr/local/share/dotnet/sdk]
  2.1.200 [/usr/local/share/dotnet/sdk]
  2.1.300 [/usr/local/share/dotnet/sdk]
  2.1.301 [/usr/local/share/dotnet/sdk]
  2.1.302 [/usr/local/share/dotnet/sdk]
  2.1.403 [/usr/local/share/dotnet/sdk]
  2.1.500 [/usr/local/share/dotnet/sdk]
  2.1.505 [/usr/local/share/dotnet/sdk]
  2.1.700 [/usr/local/share/dotnet/sdk]
  2.1.701 [/usr/local/share/dotnet/sdk]
  2.2.101 [/usr/local/share/dotnet/sdk]
  2.2.107 [/usr/local/share/dotnet/sdk]
  2.2.203 [/usr/local/share/dotnet/sdk]
  2.2.300 [/usr/local/share/dotnet/sdk]
  3.0.100-rc1-014190 [/usr/local/share/dotnet/sdk]
  3.0.100 [/usr/local/share/dotnet/sdk]
  3.1.100-preview1-014459 [/usr/local/share/dotnet/sdk]
  3.1.100-preview2-014569 [/usr/local/share/dotnet/sdk]
  3.1.100-preview3-014645 [/usr/local/share/dotnet/sdk]
  3.1.100 [/usr/local/share/dotnet/sdk]
  3.1.101 [/usr/local/share/dotnet/sdk]
  3.1.102 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.11 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.12 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.11 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.12 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview5-19227-01 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview9.19424.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-rc1.19457.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0-preview1.19508.20 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0-preview2.19528.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0-preview3.19555.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.9 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.11 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.12 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.13 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.14 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.15 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-rc1-19456-20 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0-preview1.19506.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0-preview2.19525.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0-preview3.19553.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
diez-esteban commented 4 years ago

Hi @rmarinho I'm going through the exact same error, with the exact same dev stack. I've gone through all the pages suggesting to remove the certificate from system key, run the --clean and --trust commands but nothing works.

Does anyone have further ideas on what to try next? It'd be greatly appreciated. Thank you.

javiercn commented 4 years ago

@rmarinho thanks for contacting us.

Could you check a few things? Do you have any "localhost" certificate on your keychain? (If so, assuming that it is an asp.net core generated one) Can you remove it manually? Also check on the system certificates for the same certificate and remove it from there too.

Can you run dotnet dev-certs https --check and report the exit code?

rmarinho commented 4 years ago

Hi, I removed the one I had, same error, but it does create a new one on keychain. I didn't have any on system certificates only on the login keychain.

iRuiMSFT-MBP:~ rmarinho$ dotnet dev-certs https --check
A valid HTTPS certificate was found but it may not be accessible across security partitions. Run dotnet dev-certs https to ensure it will be accessible during development.
javiercn commented 4 years ago

Can you try and run security set-key-partition-list -D localhost -S unsigned:,teamid:UBF8T346G9 <<login-keychain>> from the command-line and see if it succeeds? (replacing <<logi-keychain>> with your actual keychain path)

diez-esteban commented 4 years ago

I managed to resolve this.

I struggled with that error related to the security partitions. It's so weird. I'm still not aware as of what's the root cause of it.

javiercn commented 4 years ago
  • Run on command line dotnet tool install --global dotnet-dev-certs

You shouldn't do this, or I'm not sure it has any effect, as the dotnet-dev-certs tool is bundled with the SDK and I believe those will take preference.

This issue most likely has to do with notarization on Mac OS. Did you download the installer for Mac OS or did you use the binary distribution? I believe both should be notarized, but that can be the root of the issue

diez-esteban commented 4 years ago

I downloaded the installer for Mac OS. Maybe version 3.1 didn't include the dev-certs. Would that be the case? --check option showed me that (no certs included).

javiercn commented 4 years ago

Certs are not included, the certs are generated on the machine. Are you using macOS Catalina (10.15.4 Beta (19E242d)) ?

rmarinho commented 4 years ago

@javiercn I think my last dotnet sdk was installed by Visual Studio for Mac update system.

I m on the latest beta ( 10.15.4 Beta (19E250c)

aspnetde commented 4 years ago

Queuing in ✋

Same problem. What worked like 3 weeks ago, all of a sudden stopped. I cleaned up through dotnet dev-certs https --clean, but dotnet dev-certs https --trust then asks me to provide the password for my login.keychain-db and rejects it. I even reset it through security set-keychain-password, without success.

=== Visual Studio Community 2019 for Mac ===

Version 8.4.8 (build 2)
Installation UUID: ddc1ff0c-8d88-428e-8706-9c5852e78933
    GTK+ 2.24.23 (Raleigh theme)
    Xamarin.Mac 5.16.1.25 (issue-7441-d16-3-vsmac / 881172e73)

    Package version: 606000166

=== Mono Framework MDK ===

Runtime:
    Mono 6.6.0.166 (2019-08/d9001b5ae70) (64-bit)
    Package version: 606000166

=== Roslyn (Language Service) ===

3.4.0-beta4-19562-05+ff930dec4565e2bc424ad3bf3e22ecb20542c87d

=== .NET Core SDK ===

SDK: /usr/local/share/dotnet/sdk/3.1.102/Sdks
SDK Versions:
    3.1.102
    3.1.101
    3.1.100
    3.0.101
    3.0.100
    2.2.402
    2.1.802
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/6.6.0/lib/mono/msbuild/Current/bin/Sdks

=== .NET Core Runtime ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
    3.1.2
    3.1.1
    3.1.0
    3.0.1
    3.0.0
    2.2.7
    2.1.15
    2.1.14
    2.1.13

=== Build Information ===

Release ID: 804080002
Git revision: 4f35aa7e44fb398379e512d0bfd6f8df8d34b5ac
Build date: 2020-02-27 16:16:52+00
Build branch: release-8.4
Xamarin extensions: 4f35aa7e44fb398379e512d0bfd6f8df8d34b5ac

=== Operating System ===

Mac OS X 10.15.3
Darwin 19.3.0 Darwin Kernel Version 19.3.0
    Thu Jan  9 20:58:23 PST 2020
    root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64
javiercn commented 4 years ago

@aspnetde Are you also in the Mac OS Catalina beta?

aspnetde commented 4 years ago

Are you also in the Mac OS Catalina beta?

@javiercn Nope. Regular version.

javiercn commented 4 years ago

@aspnetde can you provide the details about the error? (console output, etc.)

You can try and run the command manually and see if that fixes the issue?

aspnetde commented 4 years ago

You can try and run the command manually and see if that fixes the issue?

As stated in my first comment, I already did that (following the docs).

Here is another failed round:

thomas@TB-MBP-2017 ~ % dotnet dev-certs https --check     
A valid HTTPS certificate was found but it may not be accessible across security partitions. Run dotnet dev-certs https to ensure it will be accessible during development.
thomas@TB-MBP-2017 ~ % dotnet dev-certs https --clean     
Cleaning HTTPS development certificates from the machine. This operation might require elevated privileges. If that is the case, a prompt for credentials will be displayed.
HTTPS development certificates successfully removed from the machine.
thomas@TB-MBP-2017 ~ % dotnet dev-certs https --check
No valid certificate found.
thomas@TB-MBP-2017 ~ % dotnet dev-certs https --trust     
A valid HTTPS certificate with a key accessible across security partitions was not found. The following command will run to fix it:
'sudo security set-key-partition-list -D localhost -S unsigned:,teamid:UBF8T346G9'
This command will make the certificate key accessible across security partitions and might prompt you for your password. For more information see: https://aka.ms/aspnetcore/2.1/troubleshootcertissues
A valid HTTPS certificate with a key accessible across security partitions was not found. The following command will run to fix it:
'sudo security set-key-partition-list -D localhost -S unsigned:,teamid:UBF8T346G9'
This command will make the certificate key accessible across security partitions and might prompt you for your password. For more information see: https://aka.ms/aspnetcore/3.1/troubleshootcertissues
Trusting the HTTPS development certificate was requested. If the certificate is not already trusted we will run the following command:
'sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain <<certificate>>'
This command might prompt you for your password to install the certificate on the system keychain.
Password:
password to unlock /Users/thomas/Library/Keychains/login.keychain-db: 
keychain: "/Users/thomas/Library/Keychains/login.keychain-db"
version: 512
class: 0x0000000F 
attributes:
    0x00000000 <uint32>=0x0000000F 
    0x00000001 <blob>="<key>"
    0x00000002 <blob>=<NULL>
    0x00000003 <uint32>=0x00000001 
    0x00000004 <uint32>=0x00000000 
    0x00000005 <uint32>=0x00000000 
    0x00000006 <blob>=0xFB53860E4AA8B4728D5B0FEF29B3090935FBD083  "\373S\206\016J\250\264r\215[\017\357)\263\011\0115\373\320\203"
    0x00000007 <blob>=<NULL>
    0x00000008 <blob>=0x7B38373139316361322D306663392D313164342D383439612D3030303530326235323132327D00  "{87191ca2-0fc9-11d4-849a-000502b52122}\000"
    0x00000009 <uint32>=0x0000002A  "\000\000\000*"
    0x0000000A <uint32>=0x00000800 
    0x0000000B <uint32>=0x00000800 
    0x0000000C <blob>=0x0000000000000000 
    0x0000000D <blob>=0x0000000000000000 
    0x0000000E <uint32>=0x00000000 
    0x0000000F <uint32>=0x00000000 
    0x00000010 <uint32>=0x00000001 
    0x00000011 <uint32>=0x00000000 
    0x00000012 <uint32>=0x00000001 
    0x00000013 <uint32>=0x00000000 
    0x00000014 <uint32>=0x00000001 
    0x00000015 <uint32>=0x00000000 
    0x00000016 <uint32>=0x00000001 
    0x00000017 <uint32>=0x00000000 
    0x00000018 <uint32>=0x00000000 
    0x00000019 <uint32>=0x00000000 
    0x0000001A <uint32>=0x00000000 
security: SecKeychainItemSetAccessWithPassword: The user name or passphrase you entered is not correct.
thomas@TB-MBP-2017 ~ % dotnet dev-certs https --check
A valid HTTPS certificate was found but it may not be accessible across security partitions. Run dotnet dev-certs https to ensure it will be accessible during development.
thomas@TB-MBP-2017 ~ % 
fleischman718 commented 4 years ago

I'm having the same issue.

when running Command: dotnet dev-certs https -c Results: A valid HTTPS certificate was found but it may not be accessible across security partitions. Run dotnet dev-certs https to ensure it will be accessible during development.

Command: dotnet dev-certs https -t -v Results: security: SecKeychainItemCopyAccess: The specified item is no longer valid. It may have been deleted from the keychain. Listing 'HTTPS' certificates on 'CurrentUser\My'. '1' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY CN=localhost - 4AED6BC2B253402E22B060BC1FB646EBEDA33D37 - 3/5/2020 9:48:35 PM - 3/5/2021 9:48:35 PM - True Checking certificates for validity. Listing valid certificates '1' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY CN=localhost - 4AED6BC2B253402E22B060BC1FB646EBEDA33D37 - 3/5/2020 9:48:35 PM - 3/5/2021 9:48:35 PM - True Listing invalid certificates '0' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY Listing 'HTTPS' certificates on 'LocalMachine\My'. '0' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY Checking certificates for validity. Listing valid certificates '0' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY Listing invalid certificates '0' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY Filtering found certificates to those with a subject equal to 'CN=localhost' '1' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY CN=localhost - 4AED6BC2B253402E22B060BC1FB646EBEDA33D37 - 3/5/2020 9:48:35 PM - 3/5/2021 9:48:35 PM - True Listing certificates excluded from consideration. '0' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY Failed to make certificate key accessible Exception message: Error making the key accessible across partitions. Something went wrong. The HTTPS developer certificate could not be created.

Mac Os Version: image

APIWT commented 4 years ago

We are also having this issue!

frozenfroze commented 4 years ago

I was facing the issue as well. Found out that the issue started with the installation of ASP Net Core SDK 3.1.102. I'm using Mac OS 10.15.3 Beta.

After I removed the SDK 3.1.102, the issue went away.

Use this to remove SDK 3.1.102 sudo rm -rf /usr/local/share/dotnet/sdk/3.1.102 sudo rm -rf /usr/local/share/dotnet/shared/Microsoft.NETCore.App/3.1.2 sudo rm -rf /usr/local/share/dotnet/shared/Microsoft.AspNetCore.App/3.1.2 sudo rm -rf /usr/local/share/dotnet/host/fxr/3.1.2

What I notice while investigating between SDK 3.1.101 and SDK 3.1.102 for the localhost cert is that 3.1.102 is missing the localhost self-signed on System and the login localhost self-signed cert is not marked as always trusted whereas SDK 3.1.101 had both login and System localhost self-signed cert and both are set at always trust for all of the trust level.

DevHwan commented 4 years ago

For me, i couldn't find /usr/local/share/dotnet/host/fxr/3.1.102. Instead i removed /usr/local/share/dotnet/host/fxr/3.1.2 and worked.

frozenfroze commented 4 years ago

For me, i couldn't find /usr/local/share/dotnet/host/fxr/3.1.102. Instead i removed /usr/local/share/dotnet/host/fxr/3.1.2 and worked.

My bad, it's 3.1.2 for the file in fxr. I'm writing based off my memory as I had already removed those files. Updated my steps

javiercn commented 4 years ago

There are many reports on this thread, so I'm going to try and give some manual steps on how to potentially address/mitigate this issue while we investigate: See here for instructions on how to remove, make accessible across partitions and trust certificates manually.

For those affected, I suggest you do as follows:

Important details for this issue

In order for us to help investigate this issue, the following information will help us:

andelizondo commented 4 years ago

I was facing the issue as well. Found out that the issue started with the installation of ASP Net Core SDK 3.1.102. I'm using Mac OS 10.15.3 Beta.

After I removed the SDK 3.1.102, the issue went away.

For me, everything was fine until I updated the SDK, but this solved the problem. I just removed that SDK version and re-generated my certificates:

sudo rm -rf /usr/local/share/dotnet/sdk/3.1.102 sudo rm -rf /usr/local/share/dotnet/shared/Microsoft.NETCore.App/3.1.2 sudo rm -rf /usr/local/share/dotnet/shared/Microsoft.AspNetCore.App/3.1.2 sudo rm -rf /usr/local/share/dotnet/host/fxr/3.1.2

Thank you @frozenfroze!!

javiercn commented 4 years ago

Important update

This is an ongoing issue in the latest SDK version (3.1.102) that we are still investigating. To workaround this issue follow these steps:

wfurt commented 4 years ago

You can find PID of securityd process and run log stream --process <PID> . That should provide additional insight into what is going on at OS level. (systemd is the processed managing access to KeyChain items) Also, this may be counterintuitive but it matters if 'dotnet' is self comes from 3.1 or not. (as all versions override same binary so sequence matters and list of available sdks is only hint) ) You can check with codesign -v -d --entitlements --extract-certificates /usr/local/share/dotnet/dotnet (or what ver path) to see if signed or unsigned binary is used.

fcbogle commented 4 years ago

I am also having serious troubles with this. I had the problem 2 weeks ago and fixed it based on the comments from @frozenfroze I added docker support which was not successful, so I smashed the project, cloned from github and now the problem is back again.

I am unable to even created the dev certificate. Any progress?

dotnet dev-certs https --check No valid certificate found.

dotnet dev-certs https --clean Cleaning HTTPS development certificates from the machine. This operation might require elevated privileges. If that is the case, a prompt for credentials will be displayed. HTTPS development certificates successfully removed from the machine.

dotnet dev-certs https A valid HTTPS certificate with a key accessible across security partitions was not found. The following command will run to fix it: 'sudo security set-key-partition-list -D localhost -S unsigned:,teamid:UBF8T346G9' This command will make the certificate key accessible across security partitions and might prompt you for your password. For more information see: https://aka.ms/aspnetcore/2.1/troubleshootcertissues A valid HTTPS certificate with a key accessible across security partitions was not found. The following command will run to fix it: 'sudo security set-key-partition-list -D localhost -S unsigned:,teamid:UBF8T346G9' This command will make the certificate key accessible across security partitions and might prompt you for your password. For more information see: https://aka.ms/aspnetcore/3.1/troubleshootcertissues Something went wrong. The HTTPS developer certificate could not be created.

dotnet --info .NET Core SDK (reflecting any global.json): Version: 3.1.201 Commit: b1768b4ae7

Runtime Environment: OS Name: Mac OS X OS Version: 10.14 OS Platform: Darwin RID: osx.10.14-x64 Base Path: /usr/local/share/dotnet/sdk/3.1.201/

Host (useful for support): Version: 3.1.3 Commit: 4a9f85e9f8

.NET Core SDKs installed: 3.0.100 [/usr/local/share/dotnet/sdk] 3.1.101 [/usr/local/share/dotnet/sdk] 3.1.200 [/usr/local/share/dotnet/sdk] 3.1.201 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed: Microsoft.AspNetCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 2.1.13 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 2.1.15 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 2.1.16 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs: https://aka.ms/dotnet-download

wfurt commented 4 years ago

make sure your KeyChain is unlocked @fcbogle. I would also recommend to use KeyChain and look for any localhost certificates.

fcbogle commented 4 years ago

Hi @wfurt thanks for your comments. I have done what you suggest (spent hours trying to debug this). Here is the output from my machine. I upgraded my macos to catalina last night. System details and key management output below:

dotnet --info .NET Core SDK (reflecting any global.json): Version: 3.1.201 Commit: b1768b4ae7

Runtime Environment: OS Name: Mac OS X OS Version: 10.15 OS Platform: Darwin RID: osx.10.15-x64 Base Path: /usr/local/share/dotnet/sdk/3.1.201/

Host (useful for support): Version: 3.1.3 Commit: 4a9f85e9f8

.NET Core SDKs installed: 3.1.201 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed: Microsoft.AspNetCore.App 3.1.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 3.1.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs: https://aka.ms/dotnet-download

======================================================================= dotnet dev-certs https --check No valid certificate found. dotnet dev-certs https --trust A valid HTTPS certificate with a key accessible across security partitions was not found. The following command will run to fix it: 'sudo security set-key-partition-list -D localhost -S unsigned:,teamid:UBF8T346G9' This command will make the certificate key accessible across security partitions and might prompt you for your password. For more information see: https://aka.ms/aspnetcore/2.1/troubleshootcertissues A valid HTTPS certificate with a key accessible across security partitions was not found. The following command will run to fix it: 'sudo security set-key-partition-list -D localhost -S unsigned:,teamid:UBF8T346G9' This command will make the certificate key accessible across security partitions and might prompt you for your password. For more information see: https://aka.ms/aspnetcore/3.1/troubleshootcertissues Trusting the HTTPS development certificate was requested. If the certificate is not already trusted we will run the following command: 'sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain <>' This command might prompt you for your password to install the certificate on the system keychain. There was an error saving the HTTPS developer certificate to the current user personal certificate store.

dotnet dev-certs https -t -v A valid HTTPS certificate with a key accessible across security partitions was not found. The following command will run to fix it: 'sudo security set-key-partition-list -D localhost -S unsigned:,teamid:UBF8T346G9' This command will make the certificate key accessible across security partitions and might prompt you for your password. For more information see: https://aka.ms/aspnetcore/2.1/troubleshootcertissues A valid HTTPS certificate with a key accessible across security partitions was not found. The following command will run to fix it: 'sudo security set-key-partition-list -D localhost -S unsigned:,teamid:UBF8T346G9' This command will make the certificate key accessible across security partitions and might prompt you for your password. For more information see: https://aka.ms/aspnetcore/3.1/troubleshootcertissues Trusting the HTTPS development certificate was requested. If the certificate is not already trusted we will run the following command: 'sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain <>' This command might prompt you for your password to install the certificate on the system keychain. Listing 'HTTPS' certificates on 'CurrentUser\My'. Listing 'HTTPS' certificates on 'LocalMachine\My'. '0' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY Checking certificates for validity. Listing valid certificates '0' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY Listing invalid certificates '0' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY Filtering found certificates to those with a subject equal to 'CN=localhost' '0' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY Listing certificates excluded from consideration. '0' found matching the criteria. SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY No valid certificates present on this machine. Trying to create one. Saving the certificate into the certificate store. Error saving the certificate in the certificate store 'CurrentUser\My'. Exception message: A default keychain could not be found. There was an error saving the HTTPS developer certificate to the current user personal certificate store.

wfurt commented 4 years ago

Any chance you do this via SSH or on system where you are no logged in in the GUI?

I was able to reproduce similar failure when I ssh 127.0.0.1 first and then run same command as same user. Now, when Keychain is locked and app needs access, OS will prompt password dialog to unlock it. That is not possible via SSH (or without GUI session) and the crypto operation will fail. If this is the case, you need to run security unlock-keychain and that will ask you for login password and it will unlock KeyChain for that session. This part is not specific to Catalina.

I also tried to generate certificate when running as "standard" user and it always fails with complain that user is not in sudden list. It may not be clear what is going on and I'm wondering if we can check this upfront or if we can get better guidance @javiercn.

wfurt commented 4 years ago

I missed keychain could not be found. from your post @fcbogle when I wrote my previous response. Can you run security list-keychains and security default-keychain ? Did you run the KeyChain Access app? You should see at least System and Login keychains there.

fcbogle commented 4 years ago

Hi @wfurt here is the output of those commands. Thank you for taking a look! security list-keychains "/Library/Keychains/System.keychain" security default-keychain security: SecKeychainCopyDefault: A default keychain could not be found.

fcbogle commented 4 years ago

Here is my Keychain: image

wfurt commented 4 years ago

I think the we are on right track. The list command does not show your login keychain and default is not set. When I run this on my system I get:

$ security list-keychain
    "/Users/furt/Library/Keychains/login.keychain-db"
    "/Library/Keychains/System.keychain"
$ security default-keychain
    "/Users/furt/Library/Keychains/login.keychain-db" 

now, it is curious that the app is showing Login keychain while the command line tool does not. If you right click on the Login keychain is there option "Make Default"? And if it is, would that change output of the commands? I did not figure out how to get location of the keychain in gui but making it default may help.

Can you also verify value of HOME environmental variable? When I unset it or point it to a "wrong" location I get same output as you.

$ HOME=/tmp/boo security list-keychain
    "/Library/Keychains/System.keychain"
$ HOME=/tmp/boo security default-keychain
security: SecKeychainCopyDefault: A default keychain could not be found.
fcbogle commented 4 years ago

Thank you! Ok, I had to fix my $HOME environment variable which is done. Here is the output now from the previous commands including the $HOME variable

Franks-iMac:~ frankbogle$ echo $HOME
/Users/frankbogle
Franks-iMac:~ frankbogle$ security list-keychain
    ""
    "/Users/frankbogle/Library/Keychains/login.keychain-db"
    "/Library/Keychains/System.keychain"
Franks-iMac:~ frankbogle$ security default-keychain
    "/Users/frankbogle/Library/Keychains/login.keychain-db"
Franks-iMac:~ frankbogle$
fcbogle commented 4 years ago

I seem to have a empty string: "" in the keychain. I can't see that in the UI

wfurt commented 4 years ago

I would backup your existing keychain and you can try to delete it with security delete-keychain. However, it should be ok to have more KeyChains so you may not bother. Is the dotnet dev-certs https --trust working now for you? Note that the HOME is used for other things as well - like package cache and other .NET files.

fcbogle commented 4 years ago

Hi @wfurt - success. Thank you for your help resolving this! I really appreciate your help!


dotnet dev-certs https -t -v
Trusting the HTTPS development certificate was requested. If the certificate is not already trusted we will run the following command:
'sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain <<certificate>>'
This command might prompt you for your password to install the certificate on the system keychain.
Listing 'HTTPS' certificates on 'CurrentUser\My'.
'1' found matching the criteria.
SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY
CN=localhost - 16CC4B15F0AD8D8B430F55ED03709C13947BD3B7 - 30/03/2020 23:27:02 - 30/03/2021 23:27:02 - True
Checking certificates for validity.
Listing valid certificates
'1' found matching the criteria.
SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY
CN=localhost - 16CC4B15F0AD8D8B430F55ED03709C13947BD3B7 - 30/03/2020 23:27:02 - 30/03/2021 23:27:02 - True
Listing invalid certificates
'0' found matching the criteria.
SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY
Listing 'HTTPS' certificates on 'LocalMachine\My'.
'1' found matching the criteria.
SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY
CN=localhost - 16CC4B15F0AD8D8B430F55ED03709C13947BD3B7 - 30/03/2020 23:27:02 - 30/03/2021 23:27:02 - False
Checking certificates for validity.
Listing valid certificates
'1' found matching the criteria.
SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY
CN=localhost - 16CC4B15F0AD8D8B430F55ED03709C13947BD3B7 - 30/03/2020 23:27:02 - 30/03/2021 23:27:02 - False
Listing invalid certificates
'0' found matching the criteria.
SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY
Filtering found certificates to those with a subject equal to 'CN=localhost'
'2' found matching the criteria.
SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY
CN=localhost - 16CC4B15F0AD8D8B430F55ED03709C13947BD3B7 - 30/03/2020 23:27:02 - 30/03/2021 23:27:02 - True
CN=localhost - 16CC4B15F0AD8D8B430F55ED03709C13947BD3B7 - 30/03/2020 23:27:02 - 30/03/2021 23:27:02 - False
Listing certificates excluded from consideration.
'0' found matching the criteria.
SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY
Found valid certificates present on the machine.
'2' found matching the criteria.
SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY
CN=localhost - 16CC4B15F0AD8D8B430F55ED03709C13947BD3B7 - 30/03/2020 23:27:02 - 30/03/2021 23:27:02 - True
CN=localhost - 16CC4B15F0AD8D8B430F55ED03709C13947BD3B7 - 30/03/2020 23:27:02 - 30/03/2021 23:27:02 - False
Selected certificate
'1' found matching the criteria.
SUBJECT - THUMBPRINT - NOT BEFORE - EXPIRES - HAS PRIVATE KEY
CN=localhost - 16CC4B15F0AD8D8B430F55ED03709C13947BD3B7 - 30/03/2020 23:27:02 - 30/03/2021 23:27:02 - True
Trying to export the certificate.
A valid HTTPS certificate is already present.```
wfurt commented 4 years ago

I'm glad it worked out. I know it is not always obvious what it is going on and Catalina did not make it easier.

gafooraa commented 4 years ago

The issue persists on Ubuntu 18.04 while using dotnet-sdk-3.1.201.

wfurt commented 4 years ago

If you have problem on Linux, open new issue @ajbozdar. All the discussion here is specific to macOS, Catalina specifically as it has distinct implementation and restrictions. Only part relevant is that Linux also depends on HOME variable to find location of user certificate store and other .NET files.

Essaadani commented 4 years ago

For me, the problem is resolved by running the following commands:

Then, I run the following command:

And finally, the HTTPS developer certificate was generated successfully

Mercally commented 4 years ago

The solution proposed by @javiercn in its batch file worked for me, just replace the password where it belonged and that worked wonderfully! Thank you.

boniyustin commented 4 years ago

The scripts from @javiercn work smoothly for me as well. Thank you. It took a day for me to solve this.

marcosxpf95 commented 4 years ago

This solve my problem https://dev.to/cesarcodes/troubleshooting-net-core-dev-certs-on-macos-179d.

javiercn commented 4 years ago

Update

Help us troubleshoot this issue

If you are experience this issue, can you try the following things and post your results here? Verify that the “localhost” identity is actually in the login keychain: security find-identity -p ssl-server -s localhost ~/Library/Keychains/Login.keychain

Run the command below manually: sudo security set-key-partition-list -D localhost -S unsigned:,teamid:UBF8T346G9 ~/Library/Keychains/Login.keychain

Verify if the key partition entry is present: security dump-keychain -a ~/Library/Keychains/Login.keychain | grep -sirB 3 -A 1 UBF8T346G9

Collect a sysdiagnose (sudo sysdiagnose) and share it with us privately (Do NOT post the file on this issue)

thales-man commented 4 years ago

in relation to #21592 I get this from my key chain Looking for identities matching "localhost"

Policy: SSL (server) Matching identities 1) 161E0C4142F4E5230E6AD64BE895E15AF57004B7 "localhost" 1 identities found

Valid identities only 1) 161E0C4142F4E5230E6AD64BE895E15AF57004B7 "localhost" 1 valid identities found when I visually check I see two certs, one is root CA

the web ui I'm trying to run still fails with cert errors

Screenshot 2020-05-07 at 21 56 13
XinHaoZhuang commented 4 years ago

I run the generate.sh , but it doesn't seem to work

./generate.sh
Generating a 2048 bit RSA private key ......+++ ............................................................+++ writing new private key to 'key.pem'

1 identity imported. Password: password to unlock /Users/apple/Library/Keychains/login.keychain-db: keychain: "/Users/apple/Library/Keychains/login.keychain-db" version: 512 class: 0x00000011 attributes: 0x00000000 = 0x00000001 ="com.apple.AppleMediaServices.mediaToken.macappstore" 0x00000002 = 0x00000003 = 0x00000004 = 0x00000005 = 0x00000006 = 0x00000007 = 0x00000008 = 0x00000009 =0x00000000 0x0000000A =0x00000000 0x0000000B =0x00000000 0x0000000C = 0x0000000D = 0x0000000E = 0x0000000F = 0x00000010 = 0x00000011 = 0x00000012 = 0x00000013 = 0x00000014 = 0x00000015 = 0x00000016 = 0x00000017 = 0x00000018 = 0x00000019 = 0x0000001A = security: SecKeychainItemCopyAccess: A missing value was detected.

dotnet --info
.NET Core SDK (reflecting any global.json): Version: 3.1.201 Commit: b1768b4ae7

Runtime Environment: OS Name: Mac OS X OS Version: 10.15 OS Platform: Darwin RID: osx.10.15-x64 Base Path: /usr/local/share/dotnet/sdk/3.1.201/

Host (useful for support): Version: 3.1.3 Commit: 4a9f85e9f8

.NET Core SDKs installed: 3.0.100 [/usr/local/share/dotnet/sdk] 3.1.200 [/usr/local/share/dotnet/sdk] 3.1.201 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed: Microsoft.AspNetCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 2.1.17 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

javiercn commented 4 years ago

@XiaroanZhang the step that is failing is the last one, the set-key-partition-list one, so you should be fine if you try to run an app (you'll have to close and reopen the browser)

thales-man commented 4 years ago

ok... so, I cleared down again. copied the workaround script into a shell file. closed all browsers and ran the script. hauled up the web ui application in VS4Mac and ran it. it requested access to the key chain (very different behaviour from all previous occasions), which I granted, and voilà... it worked.. so, my thanks to you @javiercn, but it would be great if these things weren't introduced (repeatedly) in the first place (that's a gripe). :) I'll be keeping the script for future reference... ;)

wfurt commented 4 years ago

Catalina changes were not .NET choice @thales-man. I know this can be frustrating but we are only trying to keep up.

thales-man commented 4 years ago

no, but you are a multi billion dollar organisation trying to get the other same place java was 20 years ago

XinHaoZhuang commented 4 years ago

@XiaroanZhang the step that is failing is the last one, the set-key-partition-list one, so you should be fine if you try to run an app (you'll have to close and reopen the browser)

Thank you so much.👍 It works