search

oracle / terraform-provider-oci

Terraform Oracle Cloud Infrastructure provider
https://www.terraform.io/docs/providers/oci/
Mozilla Public License 2.0
755 stars 673 forks source link

provider.oci: can not create client, bad configuration: did not find a proper configuration for private key #512

Closed GrahamHallas closed 6 years ago

GrahamHallas commented 6 years ago

Terraform Version

# Run this command to get the terraform version: $ terraform -v Terraform v0.11.7 + provider.null v1.0.0 + provider.oci v2.1.5

OCI Provider Version

# Execute the plugin directly to get the version: See above for providers

Description:

Steps show below as already read #427 and #491 so created new ssh keys with both a Pass Phrase and without, same result with both.

Have created ssh-keygen using commands ssh-keygen -t rsa -N "" -b 2048 -C "oci_terraform_ssh_key" -f ~/.ssh/id_rsa

New to Terraform and Oracle Cloud, so followed examples from Oracle Training to assign an API Key to account and create a secondary user for the trail - API Key and Fingerprint match

Terraform Plan

A self-contained terraform file that reproduces the issue. If this is not provided, your ticket may be closed. 1. source ~/env-vars cat /home/terraform/env-vars ### Authentication details export TF_VAR_tenancy_ocid="ocid1.tenancy.oc1..a" export TF_VAR_user_ocid="ocid1.user.oc1..q" export TF_VAR_fingerprint=":b3" export TF_VAR_private_key_path="/home/terraform/.oci/oci_api_key.pem"; ### Region export TF_VAR_region="eu-frankfurt-1" ### Compartment export TF_VAR_compartment_ocid="ocid1.compartment.oc1..q" ### Public/private keys used on the instance export TF_VAR_ssh_public_key=$(cat /home/terraform/.ssh/id_rsa.pub) export TF_VAR_ssh_private_key=$(cat /home/terraform/.ssh/id_rsa) echo "Loaded OCID Details" 2. Download and run terraform init Using the example files from terraform-provider-oci/docs/examples/compute/instance/ terraform@terraform-demo:~/tutorials/create_instance$ terraform init Initializing provider plugins... - Checking for available provider plugins on https://releases.hashicorp.com... - Downloading plugin for provider "null" (1.0.0)... The following providers do not have any version constraints in configuration, so the latest version was installed. To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..." constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below. * provider.null: version = "~> 1.0" * provider.oci: version = "~> 2.1" Terraform has been successfully initialized! 3. Attempt terraform plan terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. Error: Error refreshing state: 1 error(s) occurred: * provider.oci: can not create client, bad configuration: did not find a proper configuration for private key

Regards Graham

rohanbhattacharjee commented 6 years ago

The error message is indicating that the private key is the problem.

As stated in the other issues you listed above, there are certain expectations that the code has.

  1. If a passphrase is set, then it must be provided to the OCI provider using a variable TF_VAR_private_key_password
  2. The private key itself should be in the PEM format
  3. The private key path set in the TF_VAR_private_key_path should be correct

Also, make sure that the fingerprint that you provide in variable TF_VAR_fingerprint is correct. The fingerprint that you have provided looks strange, but I am guessing that you might have obfuscated it for security reasons.

GrahamHallas commented 6 years ago

Hi

Yes the details such as actual fingerprint values have been obfuscated, even though its a Cloud Trial account I am currently using, I wanted to keep it secure. so they did have "<" REMOVED ">" as part of the details, but must have been lost in the upload.

Also I am now wondering if I have this the right way, most notes see to suggest the TF_VAR_private_key_path is to the API private key file path that's associated to the fingerprint , but reading this artical: https://medium.com/@lucas_srg/how-to-onboard-a-new-customer-on-oracle-bare-metal-cloud-service-with-terraform-b716bed10146

That shows the variable set to the SSH Private Key Path , so can someone confirm which it should be??

Thanks Graham

rohanbhattacharjee commented 6 years ago

The fingerprint should match the API signing key.

https://docs.us-phoenix-1.oraclecloud.com/Content/API/Concepts/apisigningkey.htm#How3 describes how to generate the API signing keys in the PEM format, and thereafter get the corresponding fingerprint.

The SSH keys, as the name suggests, is used to ssh into the instances you deploy. When creating the instances, we inject the SSH key in the authorized_keys file in the instance (eg. https://github.com/oracle/terraform-provider-oci/blob/master/docs/examples/storage/fss/instance.tf#L10).

GrahamHallas commented 6 years ago

Thanks Rohan - After rebuilding and re-uploading the API Keys again using the same method but not via my ubuntu virtualbox it seems to have worked OK, although now have an issue with the following:

Error: Error applying plan:

3 error(s) occurred:

Creates the Storage and Network, but fails on instance creation when performing a terraform apply.

With being new to GitHub is it best to close this Issue and start a new one ??

Graham

rohanbhattacharjee commented 6 years ago

Are you trying some of the examples that we provide here - https://github.com/oracle/terraform-provider-oci/tree/master/docs/examples, or are you using your configs?

This is not an error that I would expect to see ... the shape you've chosen is a valid one. I just tried using this shape on my machine, and it works.

The compute team will be in the best position to tell us more about this error, but they will need the requestId to find the problem.

If this is a consistent error (happens every time you run), I would do the following -

  1. Turn on debug logs - TF_LOG=DEBUG OCI_GO_SDK_DEBUG=1 terraform apply
  2. Near the end of the log, you should see the service call that failed. More importantly, you should see a field called Opc-Request-Id in the response that the service sends back.
  3. Send us, or the Compute team, this request-id asking them why your shape parameter is not being accepted.

As the error message says, it could also be an authorization issue (your user account may not have the permissions to create instances etc.). The compute team can give us some definitive answers.

anandhmahal commented 6 years ago

@GrahamHallas - Are you still having issues with launching instances? Do you have service limits to launch instance in the region/AD that you are trying? If you are unsure, you can check it in OCI console, go to Identity tab and select Service Limits from the list.

cochransj commented 6 years ago

I have found that this error also occurs when using ~ in the oci_private_key_path variable.

ehaselwanter commented 5 years ago

same here

oci compute image list works with config in ~/.oci/config

but terraform plan complains about:

Error: Error refreshing state: 1 error(s) occurred:

* provider.oci: can not create client, bad configuration: did not find a proper configuration for private key

neither TF_VAR, nor other settings to provide the value works.

and it would be awesome to have something like with the aws provider where you can reuse the cli config values through a 'profile' setting

@codycushing what else is there to try? what is the root cause of this weird behaviour? the config definitely works with the cli, so it must be something in the processing of the values ... even looked at the file encoding and tested the key with openssl

 openssl rsa -in $TF_VAR_private_key_path -check
RSA key ok
writing RSA key
-----BEGIN RSA PRIVATE KEY-----
....

does it expect something else? public key with a certain naming?

sdstewar commented 4 years ago

@ehaselwanter, I'd suggest wrapping the variable in the pathexpand function when defining your provider. That should fix it and allow TF to correctly determine the path. 

provider "oci" {
  version          = ">= 3.27.0"
  region           = var.region
  tenancy_ocid     = var.tenancy_ocid
  user_ocid        = var.user_ocid
  fingerprint      = var.fingerprint
  private_key_path = pathexpand(var.private_key_path)
}
sycured commented 4 years ago

Hello, I reproduced the problem using a lot of way to provide variables.

Versions:

go version go1.14 darwin/amd64

Terraform v0.12.23
+ provider.oci v3.65.0

Error:

Error: can not create client, bad configuration: did not find a proper configuration for private key

  on main.tf line 1, in provider "oci":
   1: provider "oci" {

Code:

I used the example from OCI provider page and my variables.tf:

variable tenancy_ocid {
  default = "ocid1.tenancy.oc1.."
}
variable user_ocid {
  default = "ocid1.user.oc1.."
}
variable fingerprint {
  default = "f4:5e:.."
}
variable private_key_path {
  default = "/Users/sycured/.oci/.oci/oci_api_key.pem"
}
variable region {
  default = "sa-saopaulo-1"
}

Same problem using terraform.tfvars and export in environment.

alexng-canuck commented 4 years ago

@sycured , are you sure that your private_key_path value is correct?

You have 2 .oci folders in your path: /Users/sycured/.oci/.oci/oci_api_key.pem

Most users only have 1 .oci in their private key path (e.g. /Users/sycured/.oci/oci_api_key.pem)

The above error indicates that the provider is not able to find your key path, so it seems like a mis-configuration issue.

sycured commented 4 years ago

@alexng-canuck thank you finally, also with 1 .oci, impossible to fix it I destroyed my home and with one from scratch, it's working, I don't understand why I'd also the problem with just 1 .oci … maybe a problem with my old session

bhaveshdavda commented 4 years ago

@ehaselwanter

oci compute image list works with config in ~/.oci/config

but terraform plan complains about:

Error: Error refreshing state: 1 error(s) occurred:

* provider.oci: can not create client, bad configuration: did not find a proper configuration for private key

...

 openssl rsa -in $TF_VAR_private_key_path -check
RSA key ok
writing RSA key
-----BEGIN RSA PRIVATE KEY-----
....

I was having the exact same issue and tried all the same things. Then I tried redirecting the output of openssl rsa -in $TF_VAR_private_key_path -check and comparing it with the original key, and found the issue: my private key was simply an openssl "PRIVATE KEY" not an "RSA PRIVATE KEY".

To resolve, I simply replaced the private key pem file with the output of openssl rsa -in $TF_VAR_private_key_path -check and voilà! Terraform is now happy!

Here is how I debugged it:

  1. Littered the provider sources with debug prints but didn't get any clues
  2. Ran terraform plan under strace and noticed it was finding, opening and reading the private key pem file fine, so then why was it unhappy about the key?
  3. Found terraform-provider-oci/vendor/github.com/oracle/oci-go-sdk/common/helpers.go:PrivateKeyFromBytesWithPassword calling x509.ParsePKCS1PrivateKey which made me wonder whether that was failing because of the key file format

Any ways, just make sure your private key pem file begins with -----BEGIN RSA PRIVATE KEY----- and you should be all set

AnjanaAK commented 2 years ago

@ehaselwanter

oci compute image list works with config in ~/.oci/config but terraform plan complains about:

Error: Error refreshing state: 1 error(s) occurred:

* provider.oci: can not create client, bad configuration: did not find a proper configuration for private key

...

 openssl rsa -in $TF_VAR_private_key_path -check
RSA key ok
writing RSA key
-----BEGIN RSA PRIVATE KEY-----
....

I was having the exact same issue and tried all the same things. Then I tried redirecting the output of openssl rsa -in $TF_VAR_private_key_path -check and comparing it with the original key, and found the issue: my private key was simply an openssl "PRIVATE KEY" not an "RSA PRIVATE KEY".

To resolve, I simply replaced the private key pem file with the output of openssl rsa -in $TF_VAR_private_key_path -check and voilà! Terraform is now happy!

Here is how I debugged it:

  1. Littered the provider sources with debug prints but didn't get any clues
  2. Ran terraform plan under strace and noticed it was finding, opening and reading the private key pem file fine, so then why was it unhappy about the key?
  3. Found terraform-provider-oci/vendor/github.com/oracle/oci-go-sdk/common/helpers.go:PrivateKeyFromBytesWithPassword calling x509.ParsePKCS1PrivateKey which made me wonder whether that was failing because of the key file format

Any ways, just make sure your private key pem file begins with -----BEGIN RSA PRIVATE KEY----- and you should be all set

Hi, @bhaveshdavda @ehaselwanter

I'm also facing the issue when trying to connect to OCI using terraform provider. I was getting this error when running terraform plan:

 Error: can not create client, bad configuration: did not find a proper configuration for private key
│ 
│   with provider["registry.terraform.io/hashicorp/oci"],
│   on provider.tf line 4, in provider "oci":
│    4: provider "oci" {

The private key file is added to pipeline library secure files, and downloaded to the agent at the time of pipeline run through a task.

So checked if the file exists in the agent, and it is there. Tried printing the .pem file with cat. And the output seems fine.

And then I tried the openssl command you suggested, and it returns error:

$ openssl rsa -in $TF_VAR_private_key_path -check

unable to load Private Key
139790939505984:error:0D07209B:asn1 encoding routines:ASN1_get_object:too long:../crypto/asn1/asn1_lib.c:91:
139790939505984:error:0D068066:asn1 encoding routines:asn1_check_tlen:bad object header:../crypto/asn1/tasn_dec.c:1137:
139790939505984:error:0D07803A:asn1 encoding routines:asn1_item_embed_d2i:nested asn1 error:../crypto/asn1/tasn_dec.c:309:Type=RSAPrivateKey
139790939505984:error:04093004:rsa routines:old_rsa_priv_decode:RSA lib:../crypto/rsa/rsa_ameth.c:133:
139790939505984:error:0D07209B:asn1 encoding routines:ASN1_get_object:too long:../crypto/asn1/asn1_lib.c:91:
139790939505984:error:0D068066:asn1 encoding routines:asn1_check_tlen:bad object header:../crypto/asn1/tasn_dec.c:1137:
139790939505984:error:0D07803A:asn1 encoding routines:asn1_item_embed_d2i:nested asn1 error:../crypto/asn1/tasn_dec.c:309:Type=PKCS8_PRIV_KEY_INFO
139790939505984:error:0907B00D:PEM routines:PEM_read_bio_PrivateKey:ASN1 lib:../crypto/pem/pem_pkey.c:88:

Could you please help me to understand what went wrong here?

ehaselwanter commented 2 years ago

@AnjanaAK unfortunately we stopped using the oracle cloud

bhaveshdavda commented 2 years ago

Sounds like your pem file is corrupt

AnjanaAK commented 2 years ago

Sounds like your pem file is corrupt

@bhaveshdavda The private key file is actually added to pipeline library secure files, and downloaded to the agent at the time of pipeline run through a task. There is no such error when the api_key file is provided directly in a directory through the repository files, instead of a secure file. May be secure file adds an extra encoding to the private key file ??

briferz commented 2 years ago

Hello everyone.

I'm also facing the message Error: can not create client, bad configuration: did not find a proper configuration for the private key But in my case, I'm trying to provide the private key with the env var TF_VAR_private_key inserting the file's contents in it.

I know my private key file is fine because if I use the environment variable TF_VAR_private_key_path with the location of the private key the provider works just fine.

Is there a particular encoding or format the key contents must be provided in order for the OCI provider to be able to work properly? I have tried inserting the file into the env-var as it is in the format -----BEGIN PRIVATE KEY-----...-----END PRIVATE KEY-----, inserting the output of cat oci_api_key.pem | tr -d '\n' but it's always the same error.

aleksandr-seleznyov commented 2 years ago

Hello @briferz,

Hello everyone.

I'm also facing the message Error: can not create client, bad configuration: did not find a proper configuration for the private key But in my case, I'm trying to provide the private key with the env var TF_VAR_private_key inserting the file's contents in it.

I know my private key file is fine because if I use the environment variable TF_VAR_private_key_path with the location of the private key the provider works just fine. ....

I managed to work this around like so:

...

provider "oci" {
  region = "my_region"
  private_key = var.private_key  ### < here
}

variable "private_key" {
  default = ""
}

...

After such a change, setting the TF_VAR_private_key with the contents of a key works as expected. It looks like something is missing on the provider side. There is also no mentioning of TF_VAR_private_key_path in the official connection guide: https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/terraformproviderconfiguration.htm#:~:text=for%20more%20information.-,Environment%20Variables,-It%20is%20common

bkielbasa commented 1 year ago

@aleksandr-seleznyov it worked for me too, thanks!

brokedba commented 1 year ago

As @bhaveshdavda suggested, it is better to assign the value to TF_VAR_private_key using the opnssh command . manual copy of a multiline key into a variable is way too error prone anyway. 

export TF_VAR_private_key=`openssl rsa -in My_api_key.pem -check`
dmuiX commented 5 months ago

As @bhaveshdavda suggested, it is better to assign the value to TF_VAR_private_key using the opnssh command . manual copy of a multiline key into a variable is way too error prone anyway.

export TF_VAR_private_key=`openssl rsa -in My_api_key.pem -check`

works pretttty decent. even asks for my passphrase. love it thanks!

this is the way!

luckeyca commented 3 months ago

This is very bad recommendation even as a workaround because it won't work if only ~/.oci/config file is used which creates deviation and inconsistency from what's in oracle official documentation. Oracle developers really need to get their acts together and implement solutions properly, rather than keep introducing workarounds for their problems.

dmuiX commented 3 months ago

This is very bad recommendation even as a workaround because it won't work if only ~/.oci/config file is used which creates deviation and inconsistency from what's in oracle official documentation. Oracle developers really need to get their acts together and implement solutions properly, rather than keep introducing workarounds for their problems.

Then what is your solution? To me this seems the best we have right now.

luckeyca commented 3 months ago

This should be fixed by Oracle Terraform Provider developer. There was a similar issue with oracle go SDK. for Oracle Go SDK, Oracle fixed it by adding PKCS#8 support. See this github issue: https://github.com/oracle/oci-go-sdk/issues/242. Also the related release notes: https://github.com/oracle/oci-go-sdk/releases/tag/v22.0.0. So it's no difference for terraform provider in nature.

luckeyca commented 3 months ago

@dmuiX , I am not go developer, but did a quick search through the provider source code and did find many referenced to PKCS#1 hard-coded, but no PKCS#8. So they should be able to add it just as mentioned above. If you are go developer, you can check the source code and confirm