hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.

https://registry.terraform.io/providers/hashicorp/aws

Mozilla Public License 2.0

9.85k stars 9.2k forks source link

New service: AWS Lake Formation #9700

Closed ewbankkit closed 3 years ago

ewbankkit commented 5 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

AWS Lake Formation is now GA.

New or Affected Resource(s)

aws_XXXXX

Potential Terraform Configuration

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key.

References

Announcement. Blog post.

Requires:

https://github.com/terraform-providers/terraform-provider-aws/pull/9670

slapula commented 5 years ago

I'm also looking into writing up these resources as well.

rafiparvez commented 4 years ago

We are using AWS lakeformation for creating data lake at my organization. I am interested in contributing to this.

ewbankkit commented 4 years ago

@rafiparvez Could you please propose some resources that would match your uses cases?

dthauvin commented 4 years ago

@ewbankkit one off major features that brings AWS LakeFormation , is the capabilities to control fine-grained privilege on GlueTable ( like GRANT at column level) and s3 location with course-grained Permissions. So far to enable use case around fine grained i think we need 3 ressources:

AWS::LakeFormation::DataLakeSettings ( config between IAM policy and LakeFormation policy) AWS::LakeFormation::Permissions (represents the permissions that a principal has on catalog) AWS::LakeFormation::Resource (represents the data (Amazon S3 buckets and folders)

gmazelier commented 4 years ago

Here is a proposal to support Lake Formation settings (admins configuration), resources (de/register data locations) and permissions. Feedbacks are welcome!

tomaszdudek7 commented 4 years ago

Any ETA?

packetlag commented 4 years ago

bump for an update, please

ewbankkit commented 4 years ago

@packetlag This has been added to the August - October 2020 Roadmap. Thanks.

peramnaveen29 commented 4 years ago

14411

I guess it is related with the current AWS Lake Formation enhancement

theophilechevalier commented 4 years ago

Hi @ewbankkit, I'm willing to finish the implementation but there are already numerous open pull requests on this matter. Can you plan a review so we can fix what's missing? We're a lot to need this feature

roberth-k commented 4 years ago

Resource Links (https://github.com/terraform-providers/terraform-provider-aws/issues/15296) and CreateTableDefaultPermissions (https://github.com/terraform-providers/terraform-provider-aws/issues/14411) should be implemented as part of this epic, as they are essential for accessing cross-account Lake Formation resources using Athena and Redshift Spectrum.

YakDriver commented 3 years ago

The first release with Lake Formation support is likely to include these data sources and resources:

[x] resource/aws_lakeformation_resource (#13267)
[x] data source/aws_lakeformation_resource (#13396)
[x] resource/aws_lakeformation_data_lake_settings (#13250)
[x] data source/aws_lakeformation_data_lake_settings (#13396)
[x] resource/aws_lakeformation_permissions (#13396)
[x] data source/aws_lakeformation_permissions (#13396)

Subsequent releases (including additional contributions from community members) may include these data sources and resources:

data source/aws_lakeformation_resources (see #16845)
data source/aws_lakeformation_effective_permissions (see #16846)

priyen commented 3 years ago

@YakDriver Do you have any idea of any sort of ETA for lake formation support being available? I see lots of in flight PRs above.

YakDriver commented 3 years ago

Should be available with version 3.22!

YakDriver commented 3 years ago

Closing this since significant Lake Formation funcitonality has been merged. Remaining functionality is identified in #16845 and #16846. If there are other issues and features needed, please open a new issue! Thank you for your interest in Lake Formation!

ghost commented 3 years ago

This has been released in version 3.22.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

ghost commented 3 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!