oci_certificates_management_certificate fails with config_type="IMPORTED"

jeliker commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version and Provider Version

Terraform v1.0.9
on darwin_amd64
+ provider registry.terraform.io/hashicorp/oci v4.50.0

Affected Resource(s)

affected_resources = oci_certificates_management_certificate

Terraform Configuration Files

resource "oci_certificates_management_certificate" "the_certificate" {
  certificate_config {
    config_type     = "IMPORTED"
    cert_chain_pem  = var.cert_chain_pem
    certificate_pem = var.certificate_pem
    private_key_pem = var.private_key_pem
  }

  compartment_id = var.compartment_id
  name           = var.name
}

Debug Output

│ Error: Unsupported argument
│ 
│   on certs.tf line 4, in resource "oci_certificates_management_certificate" "the_certificate":
│    4:     cert_chain_pem = var.cert_chain_pem
│ 
│ An argument named "cert_chain_pem" is not expected here.
╵
╷
│ Error: Unsupported argument
│ 
│   on certs.tf line 5, in resource "oci_certificates_management_certificate" "the_certificate":
│    5:     certificate_pem = var.certificate_pem
│ 
│ An argument named "certificate_pem" is not expected here.
╵
╷
│ Error: Unsupported argument
│ 
│   on certs.tf line 6, in resource "oci_certificates_management_certificate" "the_certificate":
│    6:     private_key_pem = var.private_key_pem
│ 
│ An argument named "private_key_pem" is not expected here.

Panic Output

Expected Behavior

I expect IMPORTED certificate script to process properly and create a certificate resource

Actual Behavior

Certificate with config_type="IMPORTED" throws error as shown above

Steps to Reproduce

terraform plan

Important Factoids

References

Certificates IMPORT API doc https://docs.oracle.com/en-us/iaas/api/#/en/certificatesmgmt/20210224/datatypes/CreateCertificateByImportingConfigDetails

jeliker commented 2 years ago

I commented out the above and see this:

Error: expected certificate_config.0.config_type to be one of [ISSUED_BY_INTERNAL_CA MANAGED_EXTERNALLY_ISSUED_BY_INTERNAL_CA], got IMPORTED

I would call that accurate in terms of the source code but wrong in terms of the API which allows for IMPORTED cert types:

https://github.com/terraform-providers/terraform-provider-oci/blob/604b2903b48397f462658ddc22725f8cc370ffa7/oci/certificates_management_certificate_resource.go#L50-L53

https://docs.oracle.com/en-us/iaas/api/#/en/certificatesmgmt/20210224/datatypes/CreateCertificateConfigDetails
Allowed values are:
- ISSUED_BY_INTERNAL_CA (See CreateCertificateIssuedByInternalCaConfigDetails)
- MANAGED_EXTERNALLY_ISSUED_BY_INTERNAL_CA (See CreateCertificateManagedExternallyIssuedByInternalCaConfigDetails)
- IMPORTED (See CreateCertificateByImportingConfigDetails)

riyaz-sk commented 2 years ago

Please look into this issue, we are blocked because of this bug

jeliker commented 2 years ago

Have confirmed issue is still present in version 4.55.0 (#1491)

damtst commented 2 years ago

Seems it is till present on 4.67.0

mahmoudrabie commented 2 years ago

Have confirmed issue is still present in version 4.67.0

│ Error: expected certificate_config.0.config_type to be one of [ISSUED_BY_INTERNAL_CA MANAGED_EXTERNALLY_ISSUED_BY_INTERNAL_CA], got IMPORTED │ │ with oci_certificates_management_certificate....... ............. │ 6: config_type = "IMPORTED"

bluedot09 commented 2 years ago

The issue is persistent in 4.77 as well.

jeliker commented 2 years ago

This is glaringly still broken in 4.79

Error: expected certificate_config.0.config_type to be one of [ISSUED_BY_INTERNAL_CA MANAGED_EXTERNALLY_ISSUED_BY_INTERNAL_CA], got IMPORTED
│ 
│   with module.cert_hbs1_wildcard.oci_certificates_management_certificate.the_certificate,
│   on ../modules/cert/cert.tf line 1, in resource "oci_certificates_management_certificate" "the_certificate":
│    1: resource "oci_certificates_management_certificate" "the_certificate" {

Why is this seemingly ignored since initially reported over seven months back (across 29 releases of the provider)? Respectfully to the OCI certificate service, I have many (many, many) more customers using externally-generated certificates than I do using the OCI CA. Thus, supporting all types except IMPORTED seems entirely backwards.

NeilDavisNPS commented 2 years ago

Apparently, it is not broken, it is by design. If one imports the private key via terraform, it will be in the state. Despite what the documentation states, one cannot import an external certificate using terraform, so says "The Oracle"

jeliker commented 2 years ago

Respectfully, @NeilDavisNPS, that sounds like a theory to make sense of the incomplete/inconsistent behavior more than the final answer. If it were truly “by design” then why hasn’t that very simple reason been used to close this “bug” by an authoritative source over the many months it’s been open and commented on?

There have been multiple discussions here about protecting the state file (which may contain privileges information) as has long been the guidance from Hashicorp across multiple providers. Therefore, it shouldn’t be jaw-dropping to find another bit of sensitive information added to the state. If there were truly a movement to cleanse the state of confidential details, let’s talk about the DB system and TDE passwords that are there in clear text plus private key information for Load Balancer certificates, pre-shared keys for IPSec VPN, and so on.

Don’t get me wrong—I’d love to find the state file cleaned up like this but it’s very hard to accept that imported certs for the CA service is finally where this starts (notably without explanation and contrary to the documentation). I’d be very happy to find all instances of PRIVATE_KEY and DB_PASSWORD replaced with PRIVATE_KEY_DETAILS {SECRET_ID, SECRET_NAME, VAULT_ID}, DB_PASSWORD_DETAILS { . . . } and so on.

NeilDavisNPS commented 2 years ago

@jeliker , I raised a SR with Oracle, and that was the response. I too have been stumped by the inability to import external certificates using terraform. So the pipeline I have is incomplete; I have to find another way to upload the certificate (portal).

jeliker commented 2 years ago

Ah well definitely moving past anecdote at least. thank you for sharing that. Unfortunately I’ve experienced more than one SR that concludes with the technician’s opinion as to a certain behavior vs. an authoritative conclusion. Meanwhile I’ll keep badgering here until the issue here is formally closed with such explanation or until the documentation is updated to state import intentionally isn’t supported.

danielwoodz commented 1 year ago

This issue is still present in version 4.100.0.

I'll be opening a SR and having my TAM look at it, hopefully that will get it on a roadmap.

Edit: Opened SR 3-31268176681, CS is now waiting on response from Engineering. So at least this will be on their desk to triage.

jeliker commented 1 year ago

It was reported over a year ago: Very frustrating! This is a core use case for the Certificates service for my customers. Many (many, many) more are using the service for imported certificates rather than using the CA features yet.

jeliker commented 1 year ago

Here we are over two years later and still unresolved. 😕

Error: expected certificate_config.0.config_type to be one of [ISSUED_BY_INTERNAL_CA MANAGED_EXTERNALLY_ISSUED_BY_INTERNAL_CA], got IMPORTED
│ 
│   with module.cert_mycert.oci_certificates_management_certificate.the_certificate,
│   on ../terraform-oci-blueprints/modules/cert/cert.tf line 1, in resource "oci_certificates_management_certificate" "the_certificate":
│    1: resource "oci_certificates_management_certificate" "the_certificate" {

Seems like a glaring functionality gap. I've yet to have a customer using the OCI CA vs. an external CA from which certificates are generated and applied in OCI via the imported feature.

ravinitp commented 1 year ago

Thank you for reporting the issue. We have raised an internal ticket to track this. Our service engineers will get back to you.

damtst commented 1 year ago

hello,

still present on 4.121.0.

is there any way to track this issue from oracle's side?, i.e. eta for the fix regards

NeilDavisNPS commented 1 year ago

hello,

still present on 4.121.0.

is there any way to track this issue from oracle's side?, i.e. eta for the fix regards

According to Oracle, this is by design and as such, there is no fix. Even if Oracle were to admit it's an issue, they will never give an ETA.

damtst commented 1 year ago

@NeilDavisNPS , Not sure it can be tagged as "by design" if Oracle clearly supports the operation thru terraform:

https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/certificates_management_certificate

please, pay close attention mention to config_type=IMPORTED.

regards

NeilDavisNPS commented 1 year ago

@NeilDavisNPS , Not sure it can be tagged as "by design" if Oracle clearly supports the operation thru terraform:

https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/certificates_management_certificate

please, pay close attention mention to config_type=IMPORTED.

regards

I've raised this with Oracle previously (as an SR), and that was the response I was given.. unfortunately.

jeliker commented 8 months ago

Confirmed issue persists in v5.17.0. Also confirmed documentation still implies type=IMPORTED is supported.

However...

│ Error: Unsupported argument
│ 
│   on ../cert/main.tf line 10, in resource "oci_certificates_management_certificate" "the_certificate":
│   10:       cert_chain_pem = var.cert_chain_pem
│ 
│ An argument named "cert_chain_pem" is not expected here.

│ Error: Unsupported argument
│ 
│   on ../cert/main.tf line 13, in resource "oci_certificates_management_certificate" "the_certificate":
│   13:       certificate_pem = var.certificate_pem
│ 
│ An argument named "certificate_pem" is not expected here.

│ Error: Unsupported argument
│ 
│   on ../cert/main.tf line 16, in resource "oci_certificates_management_certificate" "the_certificate":
│   16:       private_key_pem = var.private_key_pem
│ 
│ An argument named "private_key_pem" is not expected here.

│ Error: Unsupported argument
│ 
│   on ../cert/main.tf line 21, in resource "oci_certificates_management_certificate" "the_certificate":
│   21:       private_key_pem_passphrase = var.private_key_pem_passphrase
│ 
│ An argument named "private_key_pem_passphrase" is not expected here.

bscarbrough commented 8 months ago

@jeliker, sadly, the only way I found around this was to fork the repo and modify the oci_certificates_management_certificate resource module, then load it into Terraform as a custom provider. I'm still doing some regression and other testing on this, but it's promising so far.

Here's the link to the file that requires modification: https://github.com/oracle/terraform-provider-oci/blob/master/internal/service/certificates_management/certificates_management_certificate_resource.go.

If you look at that file and scroll down to line 49, you can see they left "IMPORTED" out of the validation function ValidateFunc. Thus, you should just need to add "IMPORTED" as a value there, then under that section, add in the cert_chain_pem, certificate_pem, and private_key_pem variables. If you scroll down on that file to line 1338, you'll see they have the function CreateCertificateConfigDetailsToMap that should be what calls the necessary OCI API to create an imported certificate, and IMPORTED is a valid option there. There may be more required for ongoing management, things like creating new versions, etc., but so far this seems a promising short-term solution (or long-term since it seems Oracle is opposed to enabling this feature for some reason - citing security while allowing us to upload Load Balancer managed certificates, which in my book is actually less secure and more error prone... But, maybe that's just me...).

Anyway, I hope this helps, otherwise, you could also just go the route I did initially, which was to just use a null_resource and call the OCI CLI from Terraform to upload the certificates that way.

Here's what I added for those variables in the certificates_management_certificate_resource.go file in the ValidateFunc function:

"cert_chain_pem": {
    Type:     schema.TypeString,
    Optional: true,
    Computed: false,
    ForceNew: true,
},
"certificate_pem": {
    Type:     schema.TypeString,
    Optional: true,
    Computed: false,
    ForceNew: true,
},
"private_key_pem": {
    Type:     schema.TypeString,
    Optional: true,
    Computed: false,
    ForceNew: true,
},

For posterity, here's the Terraform null_resource I used:

resource "null_resource" "cert_upload_instance_principal" {
  triggers = {
    certificate      = sha256(var.certificate_info.certificate)
    certificate_name = var.certificate_name
    region           = var.region
    compartment_id   = var.compartment_id
  }

  provisioner "local-exec" {
    interpreter = ["/bin/bash", "-c"]
    command     = <<-EOF
      oci certs-mgmt certificate create-by-importing-config --region ${var.region} --auth instance_principal --compartment-id ${var.compartment_id} --name ${var.certificate_name} --cert-chain-pem '${var.certificate_info.chain}' --private-key-pem '${var.certificate_info.private_key}' --certificate-pem '${var.certificate_info.certificate}' --freeform-tags '${jsonencode(var.freeform_tags)}' --defined-tags '${jsonencode(var.defined_tags)}'
EOF
  }

  provisioner "local-exec" {
    when        = destroy
    interpreter = ["/bin/bash", "-c"]
    command     = <<-EOF
      oci certs-mgmt certificate schedule-deletion --region ${self.triggers.region} --auth instance_principal --certificate-id $(oci certs-mgmt certificate list --all --region ${self.triggers.region} --auth instance_principal --compartment-id ${self.triggers.compartment_id} --raw-output --query "data.items[?name == '${self.triggers.certificate_name}'].id | [0]")
EOF
  }
}

bscarbrough commented 8 months ago

Here's a link to my fork of the OCI Terraform provider with a modified oci_certificates_management_certificate resource allows config_type = "IMPORTED", and will actually upload the certificate to OCI Certificate Manager. ~I have not had a chance to check the Update method, but initial creation works correctly.~

Ok, It's been about a week now, and I've had a chance to test this in several scenarios, and, as it is currently, my fork of the OCI TF provider successfully adds, removes, and updates certificates via Certificate Manager.

https://github.com/bscarbrough/terraform-provider-oci/tree/master

kbens commented 4 months ago

Adding my voice to this. Confirmed issue persists in v5.28.0

Error: Unsupported argument
│ 
│   on main.tf line 26, in resource "oci_certificates_management_certificate" "certificate":
│   26:         cert_chain_pem = ""
│ 
│ An argument named "cert_chain_pem" is not expected here.
...

bscarbrough commented 4 months ago

Oh, Oracle is never going to fix this, @kbens. I use my fork in the previous post almost daily and it works like a charm, so, feel free to pull it and build a local OCI provider that'll support imported certs. I've kept it up to date (since I use it so often), and I'm working on publishing it to the registry, just need a little time to set that up.

kbens commented 4 months ago

Well, they either need to fix this or fix their documentation. It can't be working as designed and have the documentation say otherwise.

Thanks for the work on the fork. I might take a look.

bscarbrough commented 4 months ago

Oh, I don't disagree at all. I've raised it to Oracle support via our support contract and been told "We'll look in to it". So, basically, they're just going to leave it as it is and deal with frustrated users (or just watch people release versions that perform the way they "should"), I guess, since it's been this way for over 2 years now...

iolesyk commented 3 months ago

Issue persists in oracle/oci v5.36.0 It is super strange more than 2 years Oracle can't fix this issue. My try:

resource "oci_certificates_management_certificate" "test_certificate" {
    compartment_id = var.compartment_id
    name = var.cert_name
    certificate_config {
        config_type = "IMPORTED"

        certificate_pem = file(var.cert_cert_path)
        cert_chain_pem = file(var.cert_chain_path)
        private_key_pem = file(var.cert_privkey_path)
     }
}

Output:

│ Error: Unsupported argument
│
│   on main.tf line 53, in resource "oci_certificates_management_certificate" "test_certificate":
│   53:         certificate_pem = file(var.cert_cert_path)
│
│ An argument named "certificate_pem" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│   on main.tf line 54, in resource "oci_certificates_management_certificate" "test_certificate":
│   54:         cert_chain_pem = file(var.cert_chain_path)
│
│ An argument named "cert_chain_pem" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│   on main.tf line 55, in resource "oci_certificates_management_certificate" "test_certificate":
│   55:         private_key_pem = file(var.cert_privkey_path)
│
│ An argument named "private_key_pem" is not expected here.

NeilDavisNPS commented 2 months ago

Issue persists in oracle/oci v5.36.0 It is super strange more than 2 years Oracle can't fix this issue. My try:

resource "oci_certificates_management_certificate" "test_certificate" {
    compartment_id = var.compartment_id
    name = var.cert_name
    certificate_config {
        config_type = "IMPORTED"

        certificate_pem = file(var.cert_cert_path)
        cert_chain_pem = file(var.cert_chain_path)
        private_key_pem = file(var.cert_privkey_path)
     }
}

Output:

│ Error: Unsupported argument
│
│   on main.tf line 53, in resource "oci_certificates_management_certificate" "test_certificate":
│   53:         certificate_pem = file(var.cert_cert_path)
│
│ An argument named "certificate_pem" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│   on main.tf line 54, in resource "oci_certificates_management_certificate" "test_certificate":
│   54:         cert_chain_pem = file(var.cert_chain_path)
│
│ An argument named "cert_chain_pem" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│   on main.tf line 55, in resource "oci_certificates_management_certificate" "test_certificate":
│   55:         private_key_pem = file(var.cert_privkey_path)
│
│ An argument named "private_key_pem" is not expected here.

As far as Oracle as concerned, this is not a bug ; their stance is that providing this functionality is a security issue... despite their documentation. They will not be "fixing it" (or telling anyone what their plans are for the future)

oracle / terraform-provider-oci