Step creates a development certificate as byproduct and on consecutive run fails on that particular certificate

micHar commented 2 years ago

Troubleshooting

[x] I've searched discuss.bitrise.io for possible solutions.
Which version of the step is effected? 4.1.x - 4.2.x
Is the issue reproducible with the latest version? YES
Does the issue happen sporadically, or every time? EVERY TIME
Is the issue reproducible locally by following our local debug guide? NOT APPLICABLE

Issue description

I build app for ad-hoc and app store distribution with this step with api-key. On my App Store Connect I have several certificates, 8 Development and 1 for Distribution. When I run the step with this configuration, it runs fine and builds the app and it's successfully distributed to app store. However as byproduct of running that step, a new Development certificate is created via api (no idea why, but this has been confirmed in another project that uses this new step, so I guess it's by design). Unfortunately, this additional certificate ruins the next run with the error like in the logs below. Every consecutive build will fail this way until I remove this byproduct cert. At which point it runs again fine once and then the problem returns.

As a side note - in another project in our company these additional cert is created as well, but it doesn't mess up consecutive builds.

Bitrise info

Bitrise Support enabled: / NO_
Log:

+------------------------------------------------------------------------------+

| (3) xcode-archive@4                                                          |
+------------------------------------------------------------------------------+
| id: xcode-archive                                                            |
| version: 4.2.6                                                               |
| collection: https://github.com/bitrise-io/bitrise-steplib.git                |
| toolkit: go                                                                  |
| time: 2022-01-26T14:44:57Z                                                   |
+------------------------------------------------------------------------------+
|                                                                              |
INFO[14:44:57]  * [OK] Step dependency (xcode) installed, available. 
Inputs:
- distribution_method: app-store
- upload_bitcode: true
- compile_bitcode: true
- icloud_container_environment: 
- export_development_team: 
- export_options_plist_content: 
- log_formatter: xcpretty
- project_path: /Users/[REDACTED]/xxx
- scheme: xxx
- configuration: 
- output_dir: /Users/[REDACTED]/deploy
- perform_clean_action: false
- xcodebuild_options: 
- xcconfig_content: COMPILER_INDEX_STORE_ENABLE = NO
- export_all_dsyms: true
- artifact_name: 
- verbose_log: true
- cache_level: swift_packages
- automatic_code_signing: api-key
- certificate_url_list: [REDACTED]
- passphrase_list: *****
- keychain_path: /Users/[REDACTED]/Library/Keychains/login.keychain
- keychain_password: *****
- register_test_devices: false
- min_profile_validity: 0
- BITRISE_BUILD_URL: https://app.bitrise.io/build/xxx
- BITRISE_BUILD_API_TOKEN: *****
Xcode version:
Xcode 13.2.1 (Build version 13C100)
Fetching Apple Service connection
[DEBUG] GET https://app.bitrise.io/build/xxx
Bitrise Apple Developer Connection with API key found
Using Apple Service connection with API key.
Checking if log formatter (xcpretty) is installed
- xcprettyVersion: 0.3.0
Preparing code signing assets (certificates, profiles) before Archive action
Code signing asset management with xcodebuild
Reason: Automatically managed signing is enabled in Xcode for the project.
Downloading certificates from Bitrise
Downloading p12 file number 0 from [REDACTED]
[DEBUG] GET [REDACTED]
Codesign identities included:
- Serial: 133...88, Name: Apple Distribution: xxx Ltd. (xxx), Expiry: 2023-01-26 08:09:15 +0000 UTC
Valid and deduplicated certificates:
- Serial: 133...88, Name: Apple Distribution: xxxLtd. (xxx), Expiry: 2023-01-26 08:09:15 +0000 UTC
Valid certificates with type IOS_DEVELOPMENT:
Valid certificates with type IOS_DISTRIBUTION:
- Serial: 133...88, Name: Apple Distribution: xxx Ltd. (xxx), Expiry: 2023-01-26 08:09:15 +0000 UTC
Valid certificates with type IOS_DISTRIBUTION:
- Serial: 133...88, Name: Apple Distribution: xxx, Expiry: 2023-01-26 08:09:15 +0000 UTC
Valid certificates with type IOS_DISTRIBUTION
- Serial: 133...88, Name: Apple Distribution: xxx, Expiry: 2023-01-26 08:09:15 +0000 UTC 
Valid and deduplicated certificates:
- Serial: 133...88, Name: Apple Distribution: xxx, Expiry: 2023-01-26 08:09:15 +0000 UTC
Installing downloaded certificates:
- Serial: 133...88, Name: Apple Distribution: xxx, Expiry: 2023-01-26 08:09:15 +0000 UTC
Creating the Archive ...
[14:45:23] $ set -o pipefail && xcodebuild "-workspace" "/Users/[REDACTED]/git/xxx/xxx.xcworkspace" "-scheme" "xxx" "-xcconfig" "/var/folders/62/0p2cg52j6r16xjxfqch4vgt40000gn/T/256955914/temp.xcconfig" "archive" "-archivePath" "/var/folders/62/0p2cg52j6r16xjxfqch4vgt40000gn/T/xcodeArchive576485345/xxx" "-allowProvisioningUpdates" "-authenticationKeyPath" "/var/folders/62/0p2cg52j6r16xjxfqch4vgt40000gn/T/AuthKey_xxx.p8" "-authenticationKeyID" "xxx" "-authenticationKeyIssuerID" "xxx" "-destination" "generic/platform=iOS" | xcpretty
❌  error: Revoke certificate: Your account already has an Apple Development signing certificate for this machine, but its private key is not installed in your keychain. Xcode can create a new one after revoking your existing certificate. (in target 'xxx' from project 'xxx')
❌  error: No profiles for 'xxx' were found: Xcode couldn't find any iOS App Development provisioning profiles matching 'xxx'. (in target 'xxx' from project 'xxx')

lpusok commented 2 years ago

Hello @micHar, Based on the logs it seems likely that you have not uploaded a Development Certificate on the Code Signing tab on Bitrise. Can you please check and upload one if not? I think that would prevent the creation of the Apple managed Development certificate that is causing issues. If this solves the issue then we can add a check before running the Step to make sure a Development certificate is available.

micHar commented 2 years ago

Thank you for responding!

I have my Distribution certificate there. Should I add the Development certificate as well, even though I'm only building for app store / ad hoc?

Also, keep in mind that it works as long as there is no generated certificate in ASC. Its the only difference between successful and failed builds as far as I can tell.

micHar commented 2 years ago

@lpusok, it does work if I add the development cert to Bitrise. But I still don't understand why that would be necessary :)

PorterHoskins commented 2 years ago

I had the same experience. Uploading the Apple Development cert fixed it

micHar commented 2 years ago

Any info? Still happening to me

ofalvai commented 2 years ago

Hello everyone!

This behavior is in fact caused by Xcode's cloud-managed code signing and not something our step does. We are still looking into how we can work around the issue (that Xcode creates a dev cert in the background, then fails the next time).

In the meantime, the best workaround is the one already mentioned in this thread: create an Apple Development certificate manually and upload it to Bitrise so that the step can install it at runtime.

mlostekk commented 2 years ago

Thanks, that helped.

matthewbal commented 2 years ago

I had a similar issue, my step xcode build for ad-hoc started failing with:

❌ error: Choose a certificate to revoke. Your account has reached the maximum number of certificates. To create a new one, you must choose a certificate to revoke. (in target '[REDACTED]' from project '[REDACTED]')

When I checked Apple I saw that Bitrise had created nearly a dozen dev certificates with the app store connect API key.

My mistake was that I had uploaded an "iOS Development" certificate to Bitrise code signing and assumed it had worked, when in reality I should have uploaded the generic "Development" certificate. As Bitrise didn't have the "Development" certificate, it kept creating them each time we ran an ad-hoc build.

mrahn24 commented 2 years ago

Thank you @ofalvai & @matthewbal for the hint with the "Apple Developer" certificate. This fixed the automatic creation of development certificates. 🎉

But, in some of our projects (not in all), we experience a similar issue with the automatic creation of "Distribution Managed" certificates which are created by "API Key: xxxxx- ...", even if we upload the "Apple Distribution" certificates to Bitrise.

bitrise-coresteps-bot commented 1 year ago

Hello there, I'm a bot. On behalf of the community I thank you for opening this issue.

To help our human contributors focus on the most relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 90 days, so I marked it as stale.

The community would appreciate if you could check if the issue still persists. If it isn't, please close it. If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me".

If no comment left within 21 days, this issue will be closed.

micHar commented 1 year ago

Not stale

BucekJiri commented 1 year ago

@ofalvai

Any progress on this?

Uploading the development certificate workaround works fine. I would appreciate it if we could at least check the expiration date of the uploaded certificate. If the certificate expires, the step ignores it and starts creating new ones until one of the builds fail with Choose a certificate to revoke. Your account has reached the maximum number of certificates .... It would be handier if we got the error immediately when the cert expires.

bitrise-coresteps-bot commented 1 year ago

Hello there, I'm a bot. On behalf of the community I thank you for opening this issue.

To help our human contributors focus on the most relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 90 days, so I marked it as stale.

The community would appreciate if you could check if the issue still persists. If it isn't, please close it. If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me".

If no comment left within 21 days, this issue will be closed.

ghorbani-m commented 1 year ago

Hi there, I've tried all the possible ways to use Xcode archive with manual signing for app-store but does not work! It might work by auto-signing but I am trying to use manual signing! As well as I uploaded the Apple development certificate next to the Distribution certificate, but I get this error: No profiles for '[Bundle Id]' were found: Xcode couldn't find any iOS App Development provisioning profiles

BucekJiri commented 1 year ago

I believe this issue is still very relevant.

PWhittle86 commented 11 months ago

This issue is still occurring, and even the workaround is not working for me. Uploading a development certificate still causes the build to fail with the 'Your account already has an Apple Development signing certificate for this machine, but its private key is not installed in your keychain' error.

Frustrating as we would like to use Bitrise going forward, but may be forced to use XCode Cloud instead if this cannot be resolved.

BucekJiri commented 11 months ago

@PWhittle86 The workaround works fine for me. I set it up for many iOS apps. Sounds like you might not have uploaded the certificate including its private key.

You need to:

create the dev certificate on the Apple developer portal
download the certificate with its private key (this can only be done once, every other time you download it, it does not contain the private key)
import the dev certificate into your keychain
export the certificate private key from keychain (the cert needs to have "Imported private key" when you expand it in keychain)
this exports a .cer file that you upload to Bitrise

Easy peasy right? Like everything related to Apple code signing 😄

PWhittle86 commented 11 months ago

@BucekJiri thanks for your advice, but these are the steps that I've already followed! I created a brand new developer certificate on the apple developer portal and went through the usual keychain import (with private key) / export process. the only difference from what you've described is that the final certificate is in .p12 format, rather than .cer. But that's standard, from my understanding.

I'm trying again with a new development certificate, just in case there was something wrong with the first one I created.

BucekJiri commented 11 months ago

@PWhittle86 My bad, it is actually p12.

Some other things to check:

The archive step should have automatic code signing turned on and api_key set as App Store Connect authentication
This means you should have the App Store Connect API key uploaded to Bitrise
Automatic code signing should be turned on also in Xcode for the configuration you build

PWhittle86 commented 11 months ago

Please disregard my previous messages. There must have been something wrong with the development certificate I uploaded previously as now that I've created and uploaded a new cert, it's working as expected.

@BucekJiri thanks for your help!

bitrise-steplib / steps-xcode-archive