Terraform Backend with OCI Object Storage

ravinitp commented 11 months ago

Terraform Version

latest

Use Cases

A new Terraform backend utilising Oracle Cloud Infrastructure (OCI) Object Storage as the storage backend. Leveraging OCI Object Storage provides a scalable and cost-effective alternative to traditional backends, offering a robust solution for state file storage and management.

Attempted Solutions

Traditional Terraform backends, such as Amazon S3 and Azure Blob Storage, have been widely used for state file storage. However, for organisations operating within the Oracle Cloud ecosystem, OCI Object Storage presents an attractive option. This custom backend aims to facilitate Terraform deployments within OCI by seamlessly integrating with OCI Object Storage.

We aim to introduce OCI as a Terraform backend option, leveraging Object Storage. This implementation supports state lock relying on object storage alone, eliminating the need for DynamoDB when using S3-backed solutions.

Proposal

The Terraform user configures the backend using the custom backend module, specifying the OCI Object Storage details.


  backend "oci" {
    bucket    = "<mybucket>"
    object    = "<state file name>"
    namespace = "<Namespace>"

    # Other OCI authentication details
    tenancy_ocid         = "<Your Tenancy OCID>"
    user_ocid            = "<Your User OCID>"
    fingerprint          = "<Your API Key Fingerprint>"
    private_key_path     = "<Path to Your Private Key File>"
    region               = "<OCI Region>"
  }
}
--

2. During Terraform operations, the backend module interacts with OCI IAM to authenticate and obtain the necessary credentials.
3. The state file is read from or written to OCI Object Storage securely.
4. Access control policies ensure that only authorised users and services can interact with the state files.

### References

[- 32634](https://github.com/hashicorp/terraform/issues/32634)

crw commented 11 months ago

Thanks for this feature request! If you are viewing this issue and would like to indicate your interest, please use the 👍 reaction on the issue description to upvote this issue. We also welcome additional use case descriptions. Thanks again!

ravinitp commented 11 months ago

Thanks for this feature request! If you are viewing this issue and would like to indicate your interest, please use the 👍 reaction on the issue description to upvote this issue. We also welcome additional use case descriptions. Thanks again!

Hi @crw , Just wanted to know If I can start the development or should I wait for approval?

crw commented 11 months ago

I will run this past product, I didn't realize you were offering to build the backend. We have not been adding new backends in quite some time, just to set expectations, but it is always worth reviewing these policies.

MayaN2212 commented 9 months ago

Hi @crw please add this backend feature it will be beneficial for people who are using Oracle Cloud

Josephred999 commented 8 months ago

I know this is not the right platform, but is there a same backend compatibility for Dell ECS Enterprise Object Storage (Dell ECS)? If its doesnt exist, can I request the same way as original requestor did here? Hopefully, its ok to ask since I am in the same boat.

galovics commented 5 months ago

@crw is there an update on this? Thanks.

crw commented 5 months ago

No update right now. I'll raise this with product and engineering to see if we would support it. I notice that @ravinitp has as his GitHub bio "Terraform provider Developer at OCI" which is encouraging; usually the provider developer also is the codeowner for any related backend.

galovics commented 5 months ago

@crw so is it a question of supporting this or a matter of capacity to implement it? If the latter, I'm sure @ravinitp can do the implementation.

crw commented 5 months ago

It is both. When this originally came up, we had recently gone through the exercise of deprecating unmaintained backends. If @ravinitp implements this backend, the expectation is that @ravinitp would also become the CODEOWNER for this backend and support it into the future. However, every backend PR still needs to be code reviewed and approved by a core maintainer, so it does add more work for the core maintainer team and would be prioritized accordingly.

galovics commented 5 months ago

thanks @crw for clarification. @ravinitp would it be possible for you to start development on this front and become the codeowner for the backend? Thanks.

ravinitp commented 5 months ago

Sure @galovics,

I will share the ETA for this. @crw, just wanted to reconfirm: if the code looks good, satisfies all the requirements, and passes all tests, will Hashicorp allow the merge?

crw commented 5 months ago

I am currently working on getting that answer, @ravinitp. I will update this thread when I have clear guidance for you.

galovics commented 5 months ago

superb @crw, thank you.