hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.74k stars 9.1k forks source link

aws_athena_named_query not creating database #6845

Closed ghost closed 4 years ago

ghost commented 5 years ago

This issue was originally opened by @goochi1 as hashicorp/terraform#19641. It was migrated here as a result of the provider split. The original body of the issue is below.


Terraform Version

Terraform v0.11.10

Terraform Configuration Files

resource "aws_s3_bucket" "athena_bucket" { bucket = "${var.project}-athena-bucket" }

resource "aws_athena_database" "athena_database" { name = "${var.project}athenadatabase" bucket = "${aws_s3_bucket.athena_bucket.bucket}" }

resource "aws_athena_named_query" "athena_tabel_query" { name = "tabel" database = "${aws_athena_database.athena_database.name}" query = "${data.template_file.sql.rendered}" }

data "template_file" "sql" { template = "${file("${path.module}/sql.sql")}"

}


sql CREATE EXTERNAL TABLE kpmgcostanalysisathenadatabase.c_u_r( identity_line_item_id STRING, identity_time_interval STRING, bill_invoice_id STRING, bill_billing_entity STRING, bill_bill_type STRING, bill_payer_account_id STRING, bill_billing_period_start_date TIMESTAMP, bill_billing_period_end_date TIMESTAMP, line_item_usage_account_id STRING, line_item_line_item_type STRING, line_item_usage_start_date TIMESTAMP, line_item_usage_end_date TIMESTAMP, line_item_product_code STRING, line_item_usage_type STRING, line_item_operation STRING, line_item_availability_zone STRING, line_item_resource_id STRING, line_item_usage_amount DOUBLE, line_item_normalization_factor DOUBLE, line_item_normalized_usage_amount DOUBLE, line_item_currency_code STRING, line_item_unblended_rate STRING, line_item_unblended_cost DOUBLE, line_item_blended_rate STRING, line_item_blended_cost DOUBLE, line_item_line_item_description STRING, line_item_tax_type STRING, line_item_legal_entity STRING, product_product_name STRING, product_availability STRING, product_capacitystatus STRING, product_category STRING, product_clock_speed STRING, product_current_generation STRING, product_database_engine STRING, product_dedicated_ebs_throughput STRING, product_deployment_option STRING, product_description STRING, product_durability STRING, product_ecu STRING, product_edition STRING, product_engine_code STRING, product_enhanced_networking_supported STRING, product_event_type STRING, product_free_query_types STRING, product_from_location STRING, product_from_location_type STRING, product_group STRING, product_group_description STRING, product_instance_family STRING, product_instance_type STRING, product_instance_type_family STRING, product_license_model STRING, product_location STRING, product_location_type STRING, product_max_iops_burst_performance STRING, product_max_iopsvolume STRING, product_max_throughputvolume STRING, product_max_volume_size STRING, product_memory STRING, product_message_delivery_frequency STRING, product_message_delivery_order STRING, product_min_volume_size STRING, product_network_performance STRING, product_normalization_size_factor STRING, product_operating_system STRING, product_operation STRING, product_physical_processor STRING, product_pre_installed_sw STRING, product_processor_architecture STRING, product_processor_features STRING, product_product_family STRING, product_queue_type STRING, product_region STRING, product_servicecode STRING, product_servicename STRING, product_sku STRING, product_storage STRING, product_storage_class STRING, product_storage_media STRING, product_subscription_type STRING, product_tenancy STRING, product_to_location STRING, product_to_location_type STRING, product_transfer_type STRING, product_usagetype STRING, product_vcpu STRING, product_version STRING, product_volume_type STRING, pricing_lease_contract_length STRING, pricing_offering_class STRING, pricing_purchase_option STRING, pricing_rate_id STRING, pricing_public_on_demand_cost DOUBLE, pricing_public_on_demand_rate STRING, pricing_term STRING, pricing_unit STRING, reservation_amortized_upfront_cost_for_usage DOUBLE, reservation_amortized_upfront_fee_for_billing_period DOUBLE, reservation_effective_cost DOUBLE, reservation_end_time STRING, reservation_modification_status STRING, reservation_normalized_units_per_reservation STRING, reservation_number_of_reservations STRING, reservation_recurring_fee_for_usage DOUBLE, reservation_reservation_a_r_n STRING, reservation_start_time STRING, reservation_subscription_id STRING, reservation_total_reserved_normalized_units STRING, reservation_total_reserved_units STRING, reservation_units_per_reservation STRING, reservation_unused_amortized_upfront_fee_for_billing_period DOUBLE, reservation_unused_normalized_unit_quantity DOUBLE, reservation_unused_quantity DOUBLE, reservation_unused_recurring_fee DOUBLE, reservation_upfront_value DOUBLE, resource_tags_aws_autoscaling_group_name STRING, resource_tags_aws_created_by STRING, resource_tags_aws_ec2spot_fleet_request_id STRING, resource_tags_user_cost_center STRING, resource_tags_user_department STRING, resource_tags_user_environment STRING, resource_tags_user_name STRING, resource_tags_user_tag21_nov STRING, resource_tags_user_workload STRING, resource_tags_user_workload_type STRING )

PARTITIONED BY ( year STRING, month STRING )

ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' WITH SERDEPROPERTIES ( 'serialization.format' = '1' ) LOCATION "s3://costworkshop-billing/cur/WorkshopCUR/WorkshopCUR/"

Debug Output

success

Expected Behavior

Should Create a Athena Database, which is does, and run the quiery to collect data and create a table

Actual Behavior

Create a Athena Database

Steps to Reproduce

1. `terraform init`
2. `terraform apply`
hangmiao commented 5 years ago

Running into the same issue. It looks like the aws_athena_named_query is never supposed to execute the query, i.e. it is meant to create the query in Athena.

aeschright commented 4 years ago

Hi @goochi1 -- The AWS documentation is confusing to me too, but @hangmiao is correct, creating the query saves it under that name, and it can be executed separately. I'm going to close this issue, but please open a new bug report if you discover other problems. Thanks!

ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!