Closed scwall closed 3 years ago
lunika
commented
3 years ago Hi,
the readme is outdated. I will work on it today.
the bucket is referenced bucket = "marsha-terraform" and there is no variable to specify the name of the bucket in the environment file
Indeed this one is harcoded, we can create an input variable.
I will update this issue once the readme up to date.
scwall
commented
3 years ago Great thanks @lunika ! 😄
I've been digging in the last commits and I could discover that new services were being used
I was just fighting with amazon ECR and amazon lambda for the addition of the image https://gallery.ecr.aws/t3n9a8m4/marsha-lambda-medialive-routing
I'm glad to see how we proceed in the readme 😸
lunika
commented
3 years ago I was just fighting with amazon ECR and amazon lambda for the addition of the image https://gallery.ecr.aws/t3n9a8m4/marsha-lambda-medialive-routing
Yes I think it's impossible to apply all the shared_resources plan at one time. Because we need first to create the ECR image, then build and push the image. And finally deploy the lambda. I think I will split this terraform in modules and we will be able to apply each modules independently.
lunika
commented
3 years ago Can you trash everything running ./bin/shared-resources destroy and then run ./bin/shared-resources apply -target aws_ecr_repository.marsha_lambda ? This way I think it will create only the image repository. Once the image uploaded you can apply all the plan.
lunika
commented
3 years ago I made a PR (#820) updating the readme. Can you read it and tell me if it's easier for you now ?
scwall
commented
3 years ago Hello, I'm on it all day, I'm sending you my return as soon as possible today again thank you :) @lunika
scwall
commented
3 years ago Re @lunika 😄
First command :
# bfe @ marsha in ~/marsha/src/aws [10:57:10] C:1
$ ./bin/state apply
Plugin reinitialization required. Please run "terraform init".
Reason: Could not satisfy plugin requirements.
Plugins are external binaries that Terraform uses to access and manipulate
resources. The configuration provided requires plugins which can't be located,
don't satisfy the version constraints, or are otherwise incompatible.
1 error occurred:
* provider.aws: no suitable version installed
version requirements: "(any version)"
versions installed: none
Terraform automatically discovers provider requirements from your
configuration, including providers used in child modules. To see the
requirements and constraints from each module, run "terraform providers".
Error: error satisfying plugin requirementsI restarted the commands using the make init before but I still have the same mistake
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 3.20"...
- Installing hashicorp/aws v3.21.0...
- Installed hashicorp/aws v3.21.0 (signed by HashiCorp)
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
# bfe @ marsha in ~/marsha/src/aws [11:01:37]
$ ./bin/state apply Re @lunika 😄 First command :
# bfe @ marsha in ~/marsha/src/aws [10:57:10] C:1 $ ./bin/state apply Plugin reinitialization required. Please run "terraform init". Reason: Could not satisfy plugin requirements. Plugins are external binaries that Terraform uses to access and manipulate resources. The configuration provided requires plugins which can't be located, don't satisfy the version constraints, or are otherwise incompatible. 1 error occurred: * provider.aws: no suitable version installed version requirements: "(any version)" versions installed: none Terraform automatically discovers provider requirements from your configuration, including providers used in child modules. To see the requirements and constraints from each module, run "terraform providers". Error: error satisfying plugin requirementsI restarted the commands using the make init before but I still have the same mistake
Initializing provider plugins... - Finding hashicorp/aws versions matching "~> 3.20"... - Installing hashicorp/aws v3.21.0... - Installed hashicorp/aws v3.21.0 (signed by HashiCorp) Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary. # bfe @ marsha in ~/marsha/src/aws [11:01:37] $ ./bin/state apply
ok excuse use I didn't think to run before $ ./bin/state init
lunika
commented
3 years ago Yes sorry I missed the init step. I will add it in the readme.
lunika
commented
3 years ago I updated the readme with the init command. Thanks
scwall
commented
3 years ago No problem @lunika , if I can bring you a glimpse of a new user who installs without having much knowledge in Amazon services (we don't use Amazon at work). 😄
Here are the returns after creation
# bfe @ marsha in ~/marsha/src/aws [12:23:07]
$ ./bin/state apply
aws_kms_key.state_key: Refreshing state... (ID: *********)
aws_dynamodb_table.terraform-state-locks: Refreshing state... (ID: terraform_state_locks)
aws_s3_bucket.state_bucket: Refreshing state... (ID: *********)
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ aws_dynamodb_table.terraform-state-locks
id: <computed>
arn: <computed>
attribute.#: "1"
attribute.2068930648.name: "LockID"
attribute.2068930648.type: "S"
billing_mode: "PROVISIONED"
hash_key: "LockID"
name: "terraform_state_locks"
point_in_time_recovery.#: <computed>
read_capacity: "1"
server_side_encryption.#: <computed>
stream_arn: <computed>
stream_label: <computed>
stream_view_type: <computed>
write_capacity: "1"
+ aws_s3_bucket.state_bucket
id: <computed>
acceleration_status: <computed>
acl: "private"
arn: <computed>
bucket: "*********"
bucket_domain_name: <computed>
bucket_regional_domain_name: <computed>
force_destroy: "false"
hosted_zone_id: <computed>
region: "eu-west-1"
request_payer: <computed>
server_side_encryption_configuration.#: "1"
server_side_encryption_configuration.0.rule.#: "1"
server_side_encryption_configuration.0.rule.0.apply_server_side_encryption_by_default.#: "1"
server_side_encryption_configuration.0.rule.0.apply_server_side_encryption_by_default.0.kms_master_key_id: "arn:aws:kms:eu-west-1:*********:key/*********"
server_side_encryption_configuration.0.rule.0.apply_server_side_encryption_by_default.0.sse_algorithm: "aws:kms"
tags.%: "1"
tags.Name: "terraform"
versioning.#: "1"
versioning.0.enabled: "true"
versioning.0.mfa_delete: "false"
website_domain: <computed>
website_endpoint: <computed>
Plan: 2 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_s3_bucket.state_bucket: Creating...
acceleration_status: "" => "<computed>"
acl: "" => "private"
arn: "" => "<computed>"
bucket: "" => "*********"
bucket_domain_name: "" => "<computed>"
bucket_regional_domain_name: "" => "<computed>"
force_destroy: "" => "false"
hosted_zone_id: "" => "<computed>"
region: "" => "eu-west-1"
request_payer: "" => "<computed>"
server_side_encryption_configuration.#: "" => "1"
server_side_encryption_configuration.0.rule.#: "" => "1"
server_side_encryption_configuration.0.rule.0.apply_server_side_encryption_by_default.#: "" => "1"
server_side_encryption_configuration.0.rule.0.apply_server_side_encryption_by_default.0.kms_master_key_id: "" => "arn:aws:kms:eu-west-1:*********:key/*********"
server_side_encryption_configuration.0.rule.0.apply_server_side_encryption_by_default.0.sse_algorithm: "" => "aws:kms"
tags.%: "" => "1"
tags.Name: "" => "terraform"
versioning.#: "" => "1"
versioning.0.enabled: "" => "true"
versioning.0.mfa_delete: "" => "false"
website_domain: "" => "<computed>"
website_endpoint: "" => "<computed>"
aws_dynamodb_table.terraform-state-locks: Creating...
arn: "" => "<computed>"
attribute.#: "" => "1"
attribute.2068930648.name: "" => "LockID"
attribute.2068930648.type: "" => "S"
billing_mode: "" => "PROVISIONED"
hash_key: "" => "LockID"
name: "" => "terraform_state_locks"
point_in_time_recovery.#: "" => "<computed>"
read_capacity: "" => "1"
server_side_encryption.#: "" => "<computed>"
stream_arn: "" => "<computed>"
stream_label: "" => "<computed>"
stream_view_type: "" => "<computed>"
write_capacity: "" => "1"
aws_s3_bucket.state_bucket: Creation complete after 3s (ID: *********)
aws_dynamodb_table.terraform-state-locks: Creation complete after 7s (ID: terraform_state_locks)
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
Outputs:
state_bucket = *********
state_kms_key = arn:aws:kms:eu-west-1:*********:key/*********
state_locks = terraform_state_locks
# bfe @ marsha in ~/marsha/src/aws [12:24:57]
$ make init
bin/terraform init
Initializing the backend...
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 3.20"...
- Installing hashicorp/aws v3.21.0...
- Installed hashicorp/aws v3.21.0 (signed by HashiCorp)
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
bin/shared-resources init
Initializing the backend...
Backend configuration changed!
Terraform has detected that the configuration specified for the backend
has changed. Terraform will now check for existing state in the backends.
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Using previously-installed hashicorp/aws v3.21.0
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
# bfe @ marsha in ~/marsha/src/aws [12:27:18]
$ ./bin/shared-resources destroy
Do you really want to destroy all resources?
Terraform will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm.
Enter a value: yes
Destroy complete! Resources: 0 destroyed.
# bfe @ marsha in ~/marsha/src/aws [12:27:48]
$ ./bin/shared-resources apply -target aws_ecr_repository.marsha_lambda
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_ecr_repository.marsha_lambda will be created
+ resource "aws_ecr_repository" "marsha_lambda" {
+ arn = (known after apply)
+ id = (known after apply)
+ image_tag_mutability = "MUTABLE"
+ name = "marsha/lambda"
+ registry_id = (known after apply)
+ repository_url = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ ecr_lambda_arn = (known after apply)
+ ecr_lambda_url = (known after apply)
Warning: Resource targeting is in effect
You are creating a plan with the -target option, which means that the result
of this plan may not represent all of the changes requested by the current
configuration.
The -target option is not for routine use, and is provided only for
exceptional situations such as recovering from errors or mistakes, or when
Terraform specifically suggests to use it as part of an error message.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_ecr_repository.marsha_lambda: Creating...
aws_ecr_repository.marsha_lambda: Creation complete after 0s [id=marsha/lambda]
Warning: Applied changes may be incomplete
The plan was created with the -target option in effect, so some changes
requested in the configuration may have been ignored and the output values may
not be fully updated. Run the following command to verify that no other
changes are pending:
terraform plan
Note that the -target option is not suitable for routine use, and is provided
only for exceptional situations such as recovering from errors or mistakes, or
when Terraform specifically suggests to use it as part of an error message.
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
ecr_lambda_arn = arn:aws:ecr:eu-west-1:*********:repository/marsha/lambda
ecr_lambda_url = *********.dkr.ecr.eu-west-1.amazonaws.com/marsha/lambdathen I execute this step
The make init command will also create an ECR repository. Before going further you have to build and publish the lambda docker image. Unfortunately AWS doesn't allow to use a public image, so you have to host this one on a private ECR instance. Copy the output of the init command, you will use them in the next step.
Build and publish the lambda image For this step, we cooked a script to help you build, tag and deploy images. All the scripts are run from the marsha root directory.
🔧 Before you go further, you need to create ./src/aws/env.d/lambda and replace the relevant values with your own. The ECR url is available in the shared_resources terraform output you copied earlier.
You have to successively run these commands :
Build the image:
$ ./bin/lambda build Tag the image:
$ ./bin/lambda tag And then publish it:
$ ./bin/lambda publish Apply all terraform plans
I understood that I have to create a lambda file in the src/aws/env.d/ folder, I assign it ECR=*****.dkr.ecr.eu-west-1.amazonaws.com/marsha/lambda recover from my output above. But ./src/aws/env.d/lambda is in the ignore file, is there a reason? It's available in the top files marsha/env.d/lambda.dist the script ./bin/lambda is in the top folder also marsha/bin/lambda but if I try to modify the files at the top level and run the command here's what I get in return
# bfe @ marsha in ~/marsha [13:59:43]
/bin/lambda build
invalid argument ":production" for "-t, --tag" flag: invalid reference format
See 'docker build --help'.you have to copy the file ./env.d/lambda.dist in env.d/lambda and replace in this file the value with your own, you will have something like this :
AWS_ACCESS_KEY_ID=xxxxx
AWS_SECRET_ACCESS_KEY=xxxxx
AWS_REGION=eu-west-1
LAMBDA_REPOSITORY_URL=xxxxx.dkr.ecr.eu-west-1.amazonaws.com
LAMBDA_IMAGE_NAME=marsha/lambdaand the file .env.d/lambda is ignored because it contains your aws credentials. Never publish them.
scwall
commented
3 years ago ok I'm testing this, however in the documentation it is referenced in ./src/aws/env.d/lambda not ./src/env.d/lambda 😸
🔧 Before going further, you must create ./src/aws/env.d/lambda and replace the relevant values with your own. The ECR url is available in the shared_resources terraforme that you copied previously.
lunika
commented
3 years ago Oh good catch, sorry.
scwall
commented
3 years ago no problem, everything went well for the publish, but it seems that now it's stuck at the "make apply-all" level
# bfe @ marsha in ~/marsha/src/aws [15:07:20]
$ make apply-all
Error: Reference to undeclared resource
on output.tf line 10, in output "presets":
10: value = data.aws_lambda_invocation.configure_lambda_presets.result
A data resource "aws_lambda_invocation" "configure_lambda_presets" has not
been declared in the root module.
make: *** [Makefile:7: apply] Error 1
# bfe @ marsha in ~/marsha/src/aws [15:07:27] C:2
$
Indeed it was accidentally deleted in commit 52c554a. I made a commit to revert it in the PR #800
scwall
commented
3 years ago No problem 😸 Following an error I decided to delete everything and reset the amazon services to zero. I restarted the commands of the new readme one by one.
the part in readme:
Initialize your Terraform config: $ make init The make init command will also create an ECR repository. Before going further you have to build and publish the lambda docker image. Unfortunately AWS doesn't allow to use a public image, so you have to host this one on a private ECR instance. Copy the output of the init command, you will use them in the next step.
tells us that we must receive the output: with the ecr_lambda_url but we don't have this output here is what it returns:
# bfe @ marsha in ~/marsha/src/aws [15:37:19]
$ make init
bin/terraform init
Initializing the backend...
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 3.20"...
- Installing hashicorp/aws v3.21.0...
- Installed hashicorp/aws v3.21.0 (signed by HashiCorp)
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
bin/shared-resources init
Initializing the backend...
Initializing provider plugins...
- Using previously-installed hashicorp/aws v3.21.0
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.to receive the ecr_lamba_url I had to add the two commands I had to use the commands you posted above:
lunika commented yesterday • Can you trash everything running and then run ? This way I think it will create only the image repository. Once the image uploaded you can apply all the plan../bin/shared-resources destroy./bin/shared-resources apply -target aws_ecr_repository.marsha_lambda
# bfe @ marsha in ~/marsha/src/aws [15:40:03]
$ ./bin/shared-resources apply -target aws_ecr_repository.marsha_lambda
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_ecr_repository.marsha_lambda will be created
+ resource "aws_ecr_repository" "marsha_lambda" {
+ arn = (known after apply)
+ id = (known after apply)
+ image_tag_mutability = "MUTABLE"
+ name = "marsha/lambda"
+ registry_id = (known after apply)
+ repository_url = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ ecr_lambda_arn = (known after apply)
+ ecr_lambda_url = (known after apply)
Warning: Resource targeting is in effect
You are creating a plan with the -target option, which means that the result
of this plan may not represent all of the changes requested by the current
configuration.
The -target option is not for routine use, and is provided only for
exceptional situations such as recovering from errors or mistakes, or when
Terraform specifically suggests to use it as part of an error message.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_ecr_repository.marsha_lambda: Creating...
aws_ecr_repository.marsha_lambda: Creation complete after 0s [id=marsha/lambda]
Warning: Applied changes may be incomplete
The plan was created with the -target option in effect, so some changes
requested in the configuration may have been ignored and the output values may
not be fully updated. Run the following command to verify that no other
changes are pending:
terraform plan
Note that the -target option is not suitable for routine use, and is provided
only for exceptional situations such as recovering from errors or mistakes, or
when Terraform specifically suggests to use it as part of an error message.
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
ecr_lambda_arn = arn:aws:ecr:eu-west-1:*********:repository/marsha/lambda
ecr_lambda_url = *********.dkr.ecr.eu-west-1.amazonaws.com/marsha/lambdaI thought it was because the script crashed that I reinstalled everything but no it seems there is an additional error during installation
# bfe @ marsha in ~/marsha/src/aws [15:42:06]
$ make apply-all
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
<= read (data resources)
Terraform will perform the following actions:
# data.aws_lambda_invocation.configure_lambda_endpoint will be read during apply
# (config refers to values not yet known)
<= data "aws_lambda_invocation" "configure_lambda_endpoint" {
+ function_name = "default-marsha-configure"
+ id = (known after apply)
+ input = jsonencode(
{
+ Resource = "MediaConvertEndPoint"
}
)
+ result = (known after apply)
}
# data.aws_lambda_invocation.configure_lambda_presets will be read during apply
# (config refers to values not yet known)
<= data "aws_lambda_invocation" "configure_lambda_presets" {
+ function_name = "default-marsha-configure"
+ id = (known after apply)
+ input = (known after apply)
+ result = (known after apply)
}
# data.aws_lambda_invocation.invoke_migration will be read during apply
# (config refers to values not yet known)
<= data "aws_lambda_invocation" "invoke_migration" {
+ function_name = "default-marsha-migrate"
+ id = (known after apply)
+ input = jsonencode(
{
+ migrations = [
+ "0001_encode_timed_text_tracks",
]
}
)
+ result = (known after apply)
}
# aws_cloudfront_distribution.marsha_cloudfront_distribution will be created
+ resource "aws_cloudfront_distribution" "marsha_cloudfront_distribution" {
+ arn = (known after apply)
+ caller_reference = (known after apply)
+ domain_name = (known after apply)
+ enabled = true
+ etag = (known after apply)
+ hosted_zone_id = (known after apply)
+ http_version = "http2"
+ id = (known after apply)
+ in_progress_validation_batches = (known after apply)
+ is_ipv6_enabled = true
+ last_modified_time = (known after apply)
+ price_class = "PriceClass_100"
+ retain_on_delete = false
+ status = (known after apply)
+ tags = {
+ "Environment" = "default"
}
+ trusted_signers = (known after apply)
+ wait_for_deployment = true
+ default_cache_behavior {
+ allowed_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ cached_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ compress = true
+ default_ttl = 3600
+ max_ttl = 86400
+ min_ttl = 0
+ target_origin_id = "marsha-destination-origin"
+ trusted_signers = [
+ "888888888888",
]
+ viewer_protocol_policy = "redirect-to-https"
+ forwarded_values {
+ headers = [
+ "Access-Control-Request-Headers",
+ "Access-Control-Request-Method",
+ "Origin",
]
+ query_string = false
+ cookies {
+ forward = "none"
}
}
}
+ ordered_cache_behavior {
+ allowed_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ cached_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ compress = true
+ default_ttl = 3600
+ max_ttl = 86400
+ min_ttl = 0
+ path_pattern = "*/mp4/*"
+ target_origin_id = "marsha-destination-origin"
+ trusted_signers = [
+ "888888888888",
]
+ viewer_protocol_policy = "redirect-to-https"
+ forwarded_values {
+ headers = [
+ "Access-Control-Request-Headers",
+ "Access-Control-Request-Method",
+ "Origin",
]
+ query_string = true
+ cookies {
+ forward = "none"
}
}
}
+ ordered_cache_behavior {
+ allowed_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ cached_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ compress = true
+ default_ttl = 3600
+ max_ttl = 86400
+ min_ttl = 0
+ path_pattern = "*/document/*"
+ target_origin_id = "marsha-destination-origin"
+ trusted_signers = [
+ "888888888888",
]
+ viewer_protocol_policy = "redirect-to-https"
+ forwarded_values {
+ headers = [
+ "Access-Control-Request-Headers",
+ "Access-Control-Request-Method",
+ "Origin",
]
+ query_string = true
+ cookies {
+ forward = "none"
}
}
}
+ ordered_cache_behavior {
+ allowed_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ cached_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ compress = true
+ default_ttl = 3600
+ max_ttl = 86400
+ min_ttl = 0
+ path_pattern = "*/timedtext/*"
+ target_origin_id = "marsha-destination-origin"
+ trusted_signers = [
+ "888888888888",
]
+ viewer_protocol_policy = "redirect-to-https"
+ forwarded_values {
+ headers = [
+ "Access-Control-Request-Headers",
+ "Access-Control-Request-Method",
+ "Origin",
]
+ query_string = true
+ cookies {
+ forward = "none"
}
}
}
+ ordered_cache_behavior {
+ allowed_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ cached_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ compress = true
+ default_ttl = 3600
+ max_ttl = 86400
+ min_ttl = 0
+ path_pattern = "*/thumbnails/*"
+ target_origin_id = "marsha-destination-origin"
+ viewer_protocol_policy = "redirect-to-https"
+ forwarded_values {
+ headers = [
+ "Access-Control-Request-Headers",
+ "Access-Control-Request-Method",
+ "Origin",
]
+ query_string = false
+ cookies {
+ forward = "none"
}
}
}
+ ordered_cache_behavior {
+ allowed_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ cached_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ compress = true
+ default_ttl = 3600
+ max_ttl = 86400
+ min_ttl = 0
+ path_pattern = "*/cmaf/*"
+ target_origin_id = "marsha-destination-origin"
+ viewer_protocol_policy = "redirect-to-https"
+ forwarded_values {
+ headers = [
+ "Access-Control-Request-Headers",
+ "Access-Control-Request-Method",
+ "Origin",
]
+ query_string = false
+ cookies {
+ forward = "none"
}
}
}
+ ordered_cache_behavior {
+ allowed_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ cached_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ compress = true
+ default_ttl = 3600
+ max_ttl = 86400
+ min_ttl = 0
+ path_pattern = "*/previews/*"
+ target_origin_id = "marsha-destination-origin"
+ viewer_protocol_policy = "redirect-to-https"
+ forwarded_values {
+ headers = [
+ "Access-Control-Request-Headers",
+ "Access-Control-Request-Method",
+ "Origin",
]
+ query_string = false
+ cookies {
+ forward = "none"
}
}
}
+ ordered_cache_behavior {
+ allowed_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ cached_methods = [
+ "GET",
+ "HEAD",
+ "OPTIONS",
]
+ compress = true
+ default_ttl = 3600
+ max_ttl = 86400
+ min_ttl = 0
+ path_pattern = "/static/*"
+ target_origin_id = "marsha-static-origin"
+ viewer_protocol_policy = "redirect-to-https"
+ forwarded_values {
+ headers = [
+ "Access-Control-Request-Headers",
+ "Access-Control-Request-Method",
+ "Origin",
]
+ query_string = false
+ cookies {
+ forward = "none"
}
}
}
+ origin {
+ domain_name = (known after apply)
+ origin_id = "marsha-destination-origin"
+ s3_origin_config {
+ origin_access_identity = (known after apply)
}
}
+ origin {
+ domain_name = (known after apply)
+ origin_id = "marsha-static-origin"
+ s3_origin_config {
+ origin_access_identity = (known after apply)
}
}
+ restrictions {
+ geo_restriction {
+ restriction_type = "none"
}
}
+ viewer_certificate {
+ cloudfront_default_certificate = true
+ minimum_protocol_version = "TLSv1"
}
}
# aws_cloudfront_origin_access_identity.marsha_oai will be created
+ resource "aws_cloudfront_origin_access_identity" "marsha_oai" {
+ caller_reference = (known after apply)
+ cloudfront_access_identity_path = (known after apply)
+ comment = "Marsha origin for the default environment"
+ etag = (known after apply)
+ iam_arn = (known after apply)
+ id = (known after apply)
+ s3_canonical_user_id = (known after apply)
}
# aws_cloudwatch_event_rule.marsha_encode_complete_rule will be created
+ resource "aws_cloudwatch_event_rule" "marsha_encode_complete_rule" {
+ arn = (known after apply)
+ description = "Fires each time the encoding of a video source by MediaConvert is completed."
+ event_bus_name = "default"
+ event_pattern = (known after apply)
+ id = (known after apply)
+ is_enabled = true
+ name = "default-marsha-encode-complete-rule"
}
# aws_cloudwatch_event_target.marsha_encode_complete_target will be created
+ resource "aws_cloudwatch_event_target" "marsha_encode_complete_target" {
+ arn = (known after apply)
+ event_bus_name = "default"
+ id = (known after apply)
+ rule = "default-marsha-encode-complete-rule"
+ target_id = "check_foo"
}
# aws_iam_access_key.marsha_access_key will be created
+ resource "aws_iam_access_key" "marsha_access_key" {
+ encrypted_secret = (known after apply)
+ id = (known after apply)
+ key_fingerprint = (known after apply)
+ secret = (sensitive value)
+ ses_smtp_password_v4 = (sensitive value)
+ status = (known after apply)
+ user = "default-marsha"
}
# aws_iam_policy.event_rule_lambda_invoke_policy will be created
+ resource "aws_iam_policy" "event_rule_lambda_invoke_policy" {
+ arn = (known after apply)
+ description = "IAM policy for invoking a lambda from an event rule"
+ id = (known after apply)
+ name = "default-marsha-event-lambda-invoke-policy"
+ path = "/"
+ policy = (known after apply)
}
# aws_iam_policy.lambda_ecr_access_policy will be created
+ resource "aws_iam_policy" "lambda_ecr_access_policy" {
+ arn = (known after apply)
+ description = "IAM policy needed by all lambda to access ECR"
+ id = (known after apply)
+ name = "default-marsha-lambda-ecr-access-policy"
+ path = "/"
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ "ecr:SetRepositoryPolicy",
+ "ecr:GetRepositoryPolicy",
]
+ Effect = "Allow"
+ Resource = "your.ecr.image.arn/"
},
]
+ Version = "2012-10-17"
}
)
}
# aws_iam_policy.lambda_logging_policy will be created
+ resource "aws_iam_policy" "lambda_logging_policy" {
+ arn = (known after apply)
+ description = "IAM policy for logging from a lambda"
+ id = (known after apply)
+ name = "default-marsha-lambda-logging-policy"
+ path = "/"
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ "logs:CreateLogGroup",
+ "logs:CreateLogStream",
+ "logs:PutLogEvents",
]
+ Effect = "Allow"
+ Resource = "arn:aws:logs:*:*:*"
},
]
+ Version = "2012-10-17"
}
)
}
# aws_iam_policy.lambda_media_convert_policy will be created
+ resource "aws_iam_policy" "lambda_media_convert_policy" {
+ arn = (known after apply)
+ description = "IAM policy for configuring media convert from a lambda"
+ id = (known after apply)
+ name = "default-marsha-lambda-media-convert-policy"
+ path = "/"
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ "mediaconvert:CreateJob",
+ "mediaconvert:CreatePreset",
+ "mediaconvert:GetPreset",
+ "mediaconvert:UpdatePreset",
+ "mediaconvert:DescribeEndpoints",
]
+ Effect = "Allow"
+ Resource = "arn:aws:mediaconvert:*:*:*"
},
]
+ Version = "2012-10-17"
}
)
}
# aws_iam_policy.lambda_migrate_lambda_invoke_policy will be created
+ resource "aws_iam_policy" "lambda_migrate_lambda_invoke_policy" {
+ arn = (known after apply)
+ description = "IAM policy needed by lambda-migrate on S3"
+ id = (known after apply)
+ name = "default-marsha-migrate-lambda-invoke-policy"
+ path = "/"
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ "lambda:invokeAsync",
+ "lambda:invokeFunction",
]
+ Effect = "Allow"
+ Resource = "arn:aws:lambda:*:*:function:default-marsha-encode"
},
]
+ Version = "2012-10-17"
}
)
}
# aws_iam_policy.lambda_migrate_s3_access_policy will be created
+ resource "aws_iam_policy" "lambda_migrate_s3_access_policy" {
+ arn = (known after apply)
+ description = "IAM policy needed by lambda-migrate on S3"
+ id = (known after apply)
+ name = "default-marsha-migrate-lambda-s3-access-policy"
+ path = "/"
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ "s3:ListBucket",
]
+ Effect = "Allow"
+ Resource = "arn:aws:s3:::default-marsha-source"
},
]
+ Version = "2012-10-17"
}
)
}
# aws_iam_policy.lambda_pass_role_policy will be created
+ resource "aws_iam_policy" "lambda_pass_role_policy" {
+ arn = (known after apply)
+ description = "IAM policy for passing a role from a lambda"
+ id = (known after apply)
+ name = "default-marsha-lambda-pass-role-policy"
+ path = "/"
+ policy = (known after apply)
}
# aws_iam_policy.lambda_s3_access_policy will be created
+ resource "aws_iam_policy" "lambda_s3_access_policy" {
+ arn = (known after apply)
+ description = "IAM policy to read in source bucket and write in destination bucket"
+ id = (known after apply)
+ name = "default-marsha-lambda-s3-access-policy"
+ path = "/"
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ "s3:GetObject",
]
+ Effect = "Allow"
+ Resource = "arn:aws:s3:::default-marsha-source/*"
},
+ {
+ Action = [
+ "s3:GetObject",
+ "s3:PutObject",
]
+ Effect = "Allow"
+ Resource = "arn:aws:s3:::default-marsha-destination/*"
},
]
+ Version = "2012-10-17"
}
)
}
# aws_iam_policy.media_convert_s3_policy will be created
+ resource "aws_iam_policy" "media_convert_s3_policy" {
+ arn = (known after apply)
+ description = "IAM policy for accessing S3 from Media Convert"
+ id = (known after apply)
+ name = "default-marsha-media-convert-s3-policy"
+ path = "/"
+ policy = (known after apply)
}
# aws_iam_policy.medialive_custom_policy will be created
+ resource "aws_iam_policy" "medialive_custom_policy" {
+ arn = (known after apply)
+ description = "IAM policy needed to use medialive"
+ id = (known after apply)
+ name = "default-marsha-medialive-custom-policy"
+ path = "/"
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ "s3:ListBucket",
+ "s3:PutObject",
+ "s3:GetObject",
+ "s3:DeleteObject",
]
+ Effect = "Allow"
+ Resource = "*"
},
+ {
+ Action = [
+ "mediastore:ListContainers",
+ "mediastore:PutObject",
+ "mediastore:GetObject",
+ "mediastore:DeleteObject",
+ "mediastore:DescribeObject",
]
+ Effect = "Allow"
+ Resource = "*"
},
+ {
+ Action = [
+ "logs:CreateLogGroup",
+ "logs:CreateLogStream",
+ "logs:PutLogEvents",
+ "logs:DescribeLogStreams",
+ "logs:DescribeLogGroups",
]
+ Effect = "Allow"
+ Resource = "arn:aws:logs:*:*:*"
},
+ {
+ Action = [
+ "mediaconnect:ManagedDescribeFlow",
+ "mediaconnect:ManagedAddOutput",
+ "mediaconnect:ManagedRemoveOutput",
]
+ Effect = "Allow"
+ Resource = "*"
},
+ {
+ Action = [
+ "ec2:describeSubnets",
+ "ec2:describeNetworkInterfaces",
+ "ec2:createNetworkInterface",
+ "ec2:createNetworkInterfacePermission",
+ "ec2:deleteNetworkInterface",
+ "ec2:deleteNetworkInterfacePermission",
+ "ec2:describeSecurityGroups",
]
+ Effect = "Allow"
+ Resource = "*"
},
+ {
+ Action = [
+ "mediapackage:DescribeChannel",
]
+ Effect = "Allow"
+ Resource = "*"
},
]
+ Version = "2012-10-17"
}
)
}
# aws_iam_policy.ssm_read_only will be created
+ resource "aws_iam_policy" "ssm_read_only" {
+ arn = (known after apply)
+ description = "IAM policy needed access SSM in read-only mode"
+ id = (known after apply)
+ name = "default-marsha-medialive-ssm-read-only-policy"
+ path = "/"
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ "ssm:Describe*",
+ "ssm:Get*",
+ "ssm:List*",
]
+ Effect = "Allow"
+ Resource = "*"
},
]
+ Version = "2012-10-17"
}
)
}
# aws_iam_role.event_rule_role will be created
+ resource "aws_iam_role" "event_rule_role" {
+ arn = (known after apply)
+ assume_role_policy = jsonencode(
{
+ Statement = [
+ {
+ Action = "sts:AssumeRole"
+ Effect = "Allow"
+ Principal = {
+ Service = "events.amazonaws.com"
}
},
]
+ Version = "2012-10-17"
}
)
+ create_date = (known after apply)
+ force_detach_policies = false
+ id = (known after apply)
+ max_session_duration = 3600
+ name = "default-marsha-event-rule-role"
+ path = "/"
+ unique_id = (known after apply)
}
# aws_iam_role.lambda_invocation_role will be created
+ resource "aws_iam_role" "lambda_invocation_role" {
+ arn = (known after apply)
+ assume_role_policy = jsonencode(
{
+ Statement = [
+ {
+ Action = "sts:AssumeRole"
+ Effect = "Allow"
+ Principal = {
+ Service = "lambda.amazonaws.com"
}
},
]
+ Version = "2012-10-17"
}
)
+ create_date = (known after apply)
+ force_detach_policies = false
+ id = (known after apply)
+ max_session_duration = 3600
+ name = "default-marsha-lambda-invocation-role"
+ path = "/"
+ unique_id = (known after apply)
}
# aws_iam_role.lambda_medialive_invocation_role will be created
+ resource "aws_iam_role" "lambda_medialive_invocation_role" {
+ arn = (known after apply)
+ assume_role_policy = jsonencode(
{
+ Statement = [
+ {
+ Action = "sts:AssumeRole"
+ Effect = "Allow"
+ Principal = {
+ Service = "lambda.amazonaws.com"
}
},
]
+ Version = "2012-10-17"
}
)
+ create_date = (known after apply)
+ force_detach_policies = false
+ id = (known after apply)
+ max_session_duration = 3600
+ name = "default-marsha-lambda-medialive-invocation-role"
+ path = "/"
+ unique_id = (known after apply)
}
# aws_iam_role.lambda_migrate_invocation_role will be created
+ resource "aws_iam_role" "lambda_migrate_invocation_role" {
+ arn = (known after apply)
+ assume_role_policy = jsonencode(
{
+ Statement = [
+ {
+ Action = "sts:AssumeRole"
+ Effect = "Allow"
+ Principal = {
+ Service = "lambda.amazonaws.com"
}
},
]
+ Version = "2012-10-17"
}
)
+ create_date = (known after apply)
+ force_detach_policies = false
+ id = (known after apply)
+ max_session_duration = 3600
+ name = "default-marsha-lambda-migrate-invocation-role"
+ path = "/"
+ unique_id = (known after apply)
}
# aws_iam_role.media_convert_role will be created
+ resource "aws_iam_role" "media_convert_role" {
+ arn = (known after apply)
+ assume_role_policy = jsonencode(
{
+ Statement = [
+ {
+ Action = "sts:AssumeRole"
+ Effect = "Allow"
+ Principal = {
+ Service = "mediaconvert.amazonaws.com"
}
},
]
+ Version = "2012-10-17"
}
)
+ create_date = (known after apply)
+ force_detach_policies = false
+ id = (known after apply)
+ max_session_duration = 3600
+ name = "default-marsha-media-convert-role"
+ path = "/"
+ unique_id = (known after apply)
}
# aws_iam_role.medialive_access_role will be created
+ resource "aws_iam_role" "medialive_access_role" {
+ arn = (known after apply)
+ assume_role_policy = jsonencode(
{
+ Statement = [
+ {
+ Action = "sts:AssumeRole"
+ Effect = "Allow"
+ Principal = {
+ Service = "medialive.amazonaws.com"
}
},
]
+ Version = "2012-10-17"
}
)
+ create_date = (known after apply)
+ force_detach_policies = false
+ id = (known after apply)
+ max_session_duration = 3600
+ name = "default-marsha-medialive-access-role"
+ path = "/"
+ unique_id = (known after apply)
}
# aws_iam_role_policy_attachment.event_rule_lambda_invoke_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "event_rule_lambda_invoke_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-event-rule-role"
}
# aws_iam_role_policy_attachment.lambda_invocation_ecr_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "lambda_invocation_ecr_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-lambda-invocation-role"
}
# aws_iam_role_policy_attachment.lambda_logging_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "lambda_logging_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-lambda-invocation-role"
}
# aws_iam_role_policy_attachment.lambda_media_convert_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "lambda_media_convert_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-lambda-invocation-role"
}
# aws_iam_role_policy_attachment.lambda_medialive_access_ecr_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "lambda_medialive_access_ecr_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-lambda-medialive-invocation-role"
}
# aws_iam_role_policy_attachment.lambda_medialive_logging_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "lambda_medialive_logging_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-lambda-medialive-invocation-role"
}
# aws_iam_role_policy_attachment.lambda_migrate_ecr_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "lambda_migrate_ecr_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-lambda-migrate-invocation-role"
}
# aws_iam_role_policy_attachment.lambda_migrate_lambda_invoke_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "lambda_migrate_lambda_invoke_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-lambda-migrate-invocation-role"
}
# aws_iam_role_policy_attachment.lambda_migrate_logging_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "lambda_migrate_logging_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-lambda-migrate-invocation-role"
}
# aws_iam_role_policy_attachment.lambda_migrate_s3_access_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "lambda_migrate_s3_access_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-lambda-migrate-invocation-role"
}
# aws_iam_role_policy_attachment.lambda_pass_role_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "lambda_pass_role_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-lambda-invocation-role"
}
# aws_iam_role_policy_attachment.lambda_s3_access_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "lambda_s3_access_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-lambda-invocation-role"
}
# aws_iam_role_policy_attachment.media_convert_s3_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "media_convert_s3_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-media-convert-role"
}
# aws_iam_role_policy_attachment.medialive_custom_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "medialive_custom_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-medialive-access-role"
}
# aws_iam_role_policy_attachment.medialive_ssm_access_policy_attachment will be created
+ resource "aws_iam_role_policy_attachment" "medialive_ssm_access_policy_attachment" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = "default-marsha-medialive-access-role"
}
# aws_iam_user.marsha_user will be created
+ resource "aws_iam_user" "marsha_user" {
+ arn = (known after apply)
+ force_destroy = false
+ id = (known after apply)
+ name = "default-marsha"
+ path = "/"
+ unique_id = (known after apply)
}
# aws_iam_user_policy.live-streaming-policies will be created
+ resource "aws_iam_user_policy" "live-streaming-policies" {
+ id = (known after apply)
+ name = "default-marsha-live-streaming-policies"
+ policy = (known after apply)
+ user = "default-marsha"
}
# aws_lambda_function.marsha_complete_lambda will be created
+ resource "aws_lambda_function" "marsha_complete_lambda" {
+ arn = (known after apply)
+ function_name = "default-marsha-complete"
+ id = (known after apply)
+ image_uri = "your.ecr.image.name:dev"
+ invoke_arn = (known after apply)
+ last_modified = (known after apply)
+ memory_size = 128
+ package_type = "Image"
+ publish = false
+ qualified_arn = (known after apply)
+ reserved_concurrent_executions = -1
+ role = (known after apply)
+ signing_job_arn = (known after apply)
+ signing_profile_version_arn = (known after apply)
+ source_code_hash = (known after apply)
+ source_code_size = (known after apply)
+ timeout = 3
+ version = (known after apply)
+ environment {
+ variables = {
+ "DISABLE_SSL_VALIDATION" = "false"
+ "ENDPOINT" = "*********"
+ "ENV_TYPE" = "default"
+ "SHARED_SECRET" = "*********"
}
}
+ image_config {
+ command = [
+ "/var/task/lambda-complete/index.handler",
]
}
+ tracing_config {
+ mode = (known after apply)
}
}
# aws_lambda_function.marsha_configure_lambda will be created
+ resource "aws_lambda_function" "marsha_configure_lambda" {
+ arn = (known after apply)
+ function_name = "default-marsha-configure"
+ id = (known after apply)
+ image_uri = "your.ecr.image.name:dev"
+ invoke_arn = (known after apply)
+ last_modified = (known after apply)
+ memory_size = 128
+ package_type = "Image"
+ publish = false
+ qualified_arn = (known after apply)
+ reserved_concurrent_executions = -1
+ role = (known after apply)
+ signing_job_arn = (known after apply)
+ signing_profile_version_arn = (known after apply)
+ source_code_hash = (known after apply)
+ source_code_size = (known after apply)
+ timeout = 60
+ version = (known after apply)
+ environment {
+ variables = {
+ "ENV_TYPE" = "default"
}
}
+ image_config {
+ command = [
+ "/var/task/lambda-configure/index.handler",
]
}
+ tracing_config {
+ mode = (known after apply)
}
}
# aws_lambda_function.marsha_encode_lambda will be created
+ resource "aws_lambda_function" "marsha_encode_lambda" {
+ arn = (known after apply)
+ function_name = "default-marsha-encode"
+ id = (known after apply)
+ image_uri = "your.ecr.image.name:dev"
+ invoke_arn = (known after apply)
+ last_modified = (known after apply)
+ memory_size = 1536
+ package_type = "Image"
+ publish = false
+ qualified_arn = (known after apply)
+ reserved_concurrent_executions = -1
+ role = (known after apply)
+ signing_job_arn = (known after apply)
+ signing_profile_version_arn = (known after apply)
+ source_code_hash = (known after apply)
+ source_code_size = (known after apply)
+ timeout = 90
+ version = (known after apply)
+ environment {
+ variables = (known after apply)
}
+ image_config {
+ command = [
+ "/var/task/lambda-encode/index.handler",
]
}
+ tracing_config {
+ mode = (known after apply)
}
}
# aws_lambda_function.marsha_medialive_lambda will be created
+ resource "aws_lambda_function" "marsha_medialive_lambda" {
+ arn = (known after apply)
+ function_name = "default-marsha-medialive"
+ id = (known after apply)
+ image_uri = "your.ecr.image.name:dev"
+ invoke_arn = (known after apply)
+ last_modified = (known after apply)
+ memory_size = 128
+ package_type = "Image"
+ publish = false
+ qualified_arn = (known after apply)
+ reserved_concurrent_executions = -1
+ role = (known after apply)
+ signing_job_arn = (known after apply)
+ signing_profile_version_arn = (known after apply)
+ source_code_hash = (known after apply)
+ source_code_size = (known after apply)
+ timeout = 3
+ version = (known after apply)
+ environment {
+ variables = {
+ "DISABLE_SSL_VALIDATION" = "false"
+ "MARSHA_URL" = "*********"
+ "SHARED_SECRET" = "*********"
}
}
+ image_config {
+ command = [
+ "/var/task/lambda-medialive/index.handler",
]
}
+ tracing_config {
+ mode = (known after apply)
}
}
# aws_lambda_function.marsha_migrate_lambda will be created
+ resource "aws_lambda_function" "marsha_migrate_lambda" {
+ arn = (known after apply)
+ function_name = "default-marsha-migrate"
+ id = (known after apply)
+ image_uri = "your.ecr.image.name:dev"
+ invoke_arn = (known after apply)
+ last_modified = (known after apply)
+ memory_size = 128
+ package_type = "Image"
+ publish = false
+ qualified_arn = (known after apply)
+ reserved_concurrent_executions = -1
+ role = (known after apply)
+ signing_job_arn = (known after apply)
+ signing_profile_version_arn = (known after apply)
+ source_code_hash = (known after apply)
+ source_code_size = (known after apply)
+ timeout = 900
+ version = (known after apply)
+ environment {
+ variables = (known after apply)
}
+ image_config {
+ command = [
+ "/var/task/lambda-migrate/index.handler",
]
}
+ tracing_config {
+ mode = (known after apply)
}
}
# aws_lambda_permission.allow_bucket will be created
+ resource "aws_lambda_permission" "allow_bucket" {
+ action = "lambda:InvokeFunction"
+ function_name = (known after apply)
+ id = (known after apply)
+ principal = "s3.amazonaws.com"
+ source_arn = (known after apply)
+ statement_id = "AllowExecutionFromS3Bucket"
}
# aws_lambda_permission.allow_cloudwatch will be created
+ resource "aws_lambda_permission" "allow_cloudwatch" {
+ action = "lambda:InvokeFunction"
+ function_name = (known after apply)
+ id = (known after apply)
+ principal = "events.amazonaws.com"
+ source_arn = (known after apply)
+ statement_id = "AllowExecutionFromCloudWatch"
}
# aws_s3_bucket.marsha_destination will be created
+ resource "aws_s3_bucket" "marsha_destination" {
+ acceleration_status = (known after apply)
+ acl = "private"
+ arn = (known after apply)
+ bucket = "default-marsha-destination"
+ bucket_domain_name = (known after apply)
+ bucket_regional_domain_name = (known after apply)
+ force_destroy = false
+ hosted_zone_id = (known after apply)
+ id = (known after apply)
+ region = (known after apply)
+ request_payer = (known after apply)
+ tags = {
+ "Environment" = "default"
+ "Name" = "marsha-destination"
}
+ website_domain = (known after apply)
+ website_endpoint = (known after apply)
+ cors_rule {
+ allowed_headers = [
+ "*",
]
+ allowed_methods = [
+ "GET",
]
+ allowed_origins = [
+ "*",
]
+ max_age_seconds = 3600
}
+ versioning {
+ enabled = (known after apply)
+ mfa_delete = (known after apply)
}
}
# aws_s3_bucket.marsha_source will be created
+ resource "aws_s3_bucket" "marsha_source" {
+ acceleration_status = (known after apply)
+ acl = "private"
+ arn = (known after apply)
+ bucket = "default-marsha-source"
+ bucket_domain_name = (known after apply)
+ bucket_regional_domain_name = (known after apply)
+ force_destroy = false
+ hosted_zone_id = (known after apply)
+ id = (known after apply)
+ region = (known after apply)
+ request_payer = (known after apply)
+ tags = {
+ "Environment" = "default"
+ "Name" = "marsha-source"
}
+ website_domain = (known after apply)
+ website_endpoint = (known after apply)
+ cors_rule {
+ allowed_headers = [
+ "*",
]
+ allowed_methods = [
+ "POST",
]
+ allowed_origins = [
+ "*",
]
+ max_age_seconds = 3600
}
+ versioning {
+ enabled = (known after apply)
+ mfa_delete = (known after apply)
}
}
# aws_s3_bucket.marsha_static will be created
+ resource "aws_s3_bucket" "marsha_static" {
+ acceleration_status = (known after apply)
+ acl = "private"
+ arn = (known after apply)
+ bucket = "default-marsha-static"
+ bucket_domain_name = (known after apply)
+ bucket_regional_domain_name = (known after apply)
+ force_destroy = false
+ hosted_zone_id = (known after apply)
+ id = (known after apply)
+ region = (known after apply)
+ request_payer = (known after apply)
+ tags = {
+ "Environment" = "default"
+ "Name" = "marsha-static"
}
+ website_domain = (known after apply)
+ website_endpoint = (known after apply)
+ cors_rule {
+ allowed_headers = [
+ "*",
]
+ allowed_methods = [
+ "GET",
]
+ allowed_origins = [
+ "*",
]
+ max_age_seconds = 3600
}
+ versioning {
+ enabled = (known after apply)
+ mfa_delete = (known after apply)
}
}
# aws_s3_bucket_notification.marsha_source_bucket_notification will be created
+ resource "aws_s3_bucket_notification" "marsha_source_bucket_notification" {
+ bucket = (known after apply)
+ id = (known after apply)
+ lambda_function {
+ events = [
+ "s3:ObjectCreated:*",
]
+ id = (known after apply)
+ lambda_function_arn = (known after apply)
}
}
# aws_s3_bucket_policy.marsha_destination_bucket_policy will be created
+ resource "aws_s3_bucket_policy" "marsha_destination_bucket_policy" {
+ bucket = (known after apply)
+ id = (known after apply)
+ policy = (known after apply)
}
# aws_s3_bucket_policy.marsha_source_bucket_policy will be created
+ resource "aws_s3_bucket_policy" "marsha_source_bucket_policy" {
+ bucket = (known after apply)
+ id = (known after apply)
+ policy = (known after apply)
}
# aws_s3_bucket_policy.marsha_static_bucket_policy will be created
+ resource "aws_s3_bucket_policy" "marsha_static_bucket_policy" {
+ bucket = (known after apply)
+ id = (known after apply)
+ policy = (known after apply)
}
Plan: 53 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_iam_policy.lambda_media_convert_policy: Creating...
aws_iam_role.lambda_migrate_invocation_role: Creating...
aws_iam_policy.ssm_read_only: Creating...
aws_iam_role.event_rule_role: Creating...
aws_iam_role.media_convert_role: Creating...
aws_iam_user.marsha_user: Creating...
aws_iam_role.medialive_access_role: Creating...
aws_s3_bucket.marsha_destination: Creating...
aws_iam_policy.lambda_logging_policy: Creating...
aws_s3_bucket.marsha_static: Creating...
aws_iam_policy.lambda_ecr_access_policy: Creating...
aws_iam_policy.medialive_custom_policy: Creating...
aws_cloudfront_origin_access_identity.marsha_oai: Creating...
aws_iam_role.lambda_invocation_role: Creating...
aws_iam_role.lambda_medialive_invocation_role: Creating...
aws_s3_bucket.marsha_source: Creating...
aws_cloudfront_origin_access_identity.marsha_oai: Creation complete after 2s [id=E279D9V7YRLZGZ]
aws_s3_bucket.marsha_destination: Creation complete after 3s [id=default-marsha-destination]
aws_s3_bucket.marsha_static: Creation complete after 3s [id=default-marsha-static]
aws_cloudfront_distribution.marsha_cloudfront_distribution: Creating...
aws_s3_bucket.marsha_source: Creation complete after 4s [id=default-marsha-source]
aws_iam_policy.lambda_migrate_s3_access_policy: Creating...
aws_cloudwatch_event_rule.marsha_encode_complete_rule: Creating...
aws_iam_policy.media_convert_s3_policy: Creating...
aws_iam_policy.lambda_s3_access_policy: Creating...
aws_cloudwatch_event_rule.marsha_encode_complete_rule: Creation complete after 0s [id=default-marsha-encode-complete-rule]
Error: Error creating IAM Role default-marsha-media-convert-role: EntityAlreadyExists: Role with name default-marsha-media-convert-role already exists.
status code: 409, request id: *********
Error: Error creating IAM policy default-marsha-lambda-logging-policy: EntityAlreadyExists: A policy called default-marsha-lambda-logging-policy already exists. Duplicate names are not allowed.
status code: 409, request id: *********
Error: Error creating IAM Role default-marsha-lambda-migrate-invocation-role: EntityAlreadyExists: Role with name default-marsha-lambda-migrate-invocation-role already exists.
status code: 409, request id: *********
Error: Error creating IAM policy default-marsha-lambda-ecr-access-policy: MalformedPolicyDocument: Resource your.ecr.image.arn/ must be in ARN format or "*".
status code: 400, request id: *********
Error: Error creating IAM policy default-marsha-migrate-lambda-s3-access-policy: EntityAlreadyExists: A policy called default-marsha-migrate-lambda-s3-access-policy already exists. Duplicate names are not allowed.
status code: 409, request id: *********
Error: Error creating IAM User default-marsha: EntityAlreadyExists: User with name default-marsha already exists.
status code: 409, request id: *********
Error: Error creating IAM policy default-marsha-lambda-media-convert-policy: EntityAlreadyExists: A policy called default-marsha-lambda-media-convert-policy already exists. Duplicate names are not allowed.
status code: 409, request id: *********
Error: Error creating IAM Role default-marsha-medialive-access-role: EntityAlreadyExists: Role with name default-marsha-medialive-access-role already exists.
status code: 409, request id: *********
Error: Error creating IAM policy default-marsha-medialive-ssm-read-only-policy: EntityAlreadyExists: A policy called default-marsha-medialive-ssm-read-only-policy already exists. Duplicate names are not allowed.
status code: 409, request id: *********
Error: Error creating IAM Role default-marsha-lambda-invocation-role: EntityAlreadyExists: Role with name default-marsha-lambda-invocation-role already exists.
status code: 409, request id: *********
Error: error creating Lambda Function: InvalidParameterValueException: Source image public.ecr.aws//********//marsha-lambda-medialive-routing:dev is not valid. Provide a valid source image.
{
RespMetadata: {
StatusCode: 400,
RequestID: "*********"
},
Message_: "Source image public.ecr.aws//********//marsha-lambda-medialive-routing:dev is not valid. Provide a valid source image.",
Type: "User"
}
Error: error creating CloudFront Distribution: TrustedSignerDoesNotExist: One or more of your trusted signers do not exist.
status code: 400, request id: *********
Error: error creating Lambda Function: InvalidParameterValueException: Source image public.ecr.aws/*****/marsha-lambda-medialive-routing:dev is not valid. Provide a valid source image.
{
RespMetadata: {
StatusCode: 400,
RequestID: "*********"
},
Message_: "Source image public.ecr.aws/********/marsha-lambda-medialive-routing:dev is not valid. Provide a valid source image.",
Type: "User"
}
Error: error creating Lambda Function: InvalidParameterValueException: Source image public.ecr.aws//********/marsha-lambda-medialive-routing:dev is not valid. Provide a valid source image.
{
RespMetadata: {
StatusCode: 400,
RequestID: "*********"
},
Message_: "Source image public.ecr.aws//********/marsha-lambda-medialive-routing:dev is not valid. Provide a valid source image.",
Type: "User"
}
make: *** [Makefile:7: apply] Error 1
# bfe @ marsha in ~/marsha/src/aws [15:42:26] C:2
ok I have made a patch and also deleted the IAM policies, at the image level I have added my image that I generated and not the generic name of the public image:
TF_VAR_lambda_image_name=*******.dkr.ecr.eu-west-1.amazonaws.com/marsha/lambda
TF_VAR_lambda_image_tag=production
TF_VAR_ecr_lambda_marsha_arn=arn:aws:ecr:eu-west-1:******:repository/marsha/lambdahere's the new error I'm coming out now
Error: Lambda function (default-marsha-configure) returned error: ({"errorType":"AccessDeniedException","errorMessage":"User: arn:aws:sts::*********:assumed-role/default-marsha-lambda-invocation-role/default-marsha-configure is not authorized to perform: mediaconvert:DescribeEndpoints on resource: arn:aws:mediaconvert:eu-west-1:*********:endpoints/*","trace":["AccessDeniedException: User: arn:aws:sts::*********:assumed-role/default-marsha-lambda-invocation-role/default-marsha-configure is not authorized to perform: mediaconvert:DescribeEndpoints on resource: arn:aws:mediaconvert:eu-west-1:*********:endpoints/*"," at Object.extractError (/var/task/lambda-configure/node_modules/aws-sdk/lib/protocol/json.js:52:27)"," at Request.extractError (/var/task/lambda-configure/node_modules/aws-sdk/lib/protocol/rest_json.js:55:8)"," at Request.callListeners (/var/task/lambda-configure/node_modules/aws-sdk/lib/sequential_executor.js:106:20)"," at Request.emit (/var/task/lambda-configure/node_modules/aws-sdk/lib/sequential_executor.js:78:10)"," at Request.emit (/var/task/lambda-configure/node_modules/aws-sdk/lib/request.js:688:14)"," at Request.transition (/var/task/lambda-configure/node_modules/aws-sdk/lib/request.js:22:10)"," at AcceptorStateMachine.runTo (/var/task/lambda-configure/node_modules/aws-sdk/lib/state_machine.js:14:12)"," at /var/task/lambda-configure/node_modules/aws-sdk/lib/state_machine.js:26:10"," at Request.<anonymous> (/var/task/lambda-configure/node_modules/aws-sdk/lib/request.js:38:9)"," at Request.<anonymous> (/var/task/lambda-configure/node_modules/aws-sdk/lib/request.js:690:12)"]})
`
lunika
commented
3 years ago Ok I was replying you to manually delete the resources already existing.
Can you tell me if the policy default-marsha-lambda-media-convert-policy exists ? And if this policy is attached to the role default-marsha-lambda-invocation-role.
policies can be found here https://console.aws.amazon.com/iam/home?region=eu-west-1#/policies roles can be found here https://console.aws.amazon.com/iam/home?region=eu-west-1#/roles
scwall
commented
3 years ago ok it worked much better I deleted the lambda functions and I deleted all the roles and policy indicated, then I restarted everything. It took 10 minutes and then I received new errors.
What's weird is that I removed all the IAM policies but it seems that it still blocks after a while
Error: Error creating IAM policy default-marsha-event-lambda-invoke-policy: EntityAlreadyExists: A policy called default-marsha-event-lambda-invoke-policy already exists. Duplicate names are not allowed.
status code: 409, request id: db5b098a-a58f-4077-aed0-f081b8d22cae
Error: Error creating IAM policy default-marsha-lambda-pass-role-policy: EntityAlreadyExists: A policy called default-marsha-lambda-pass-role-policy already exists. Duplicate names are not allowed.
status code: 409, request id: dea9d35f-d5dc-4eec-9c4f-a1caf7379efa
Error: Error creating IAM policy default-marsha-medialive-ssm-read-only-policy: EntityAlreadyExists: A policy called default-marsha-medialive-ssm-read-only-policy already exists. Duplicate names are not allowed.
status code: 409, request id: 9f7250bc-dc6c-4c85-a5e9-c509ff6d61fc
Error: Lambda function (default-marsha-configure) returned error: ({"errorType":"Error","errorMessage":"ENOENT: no such file or directory, open './presets/cmaf_audio_aac_64kbps.json'","trace":["Error: ENOENT: no such file or directory, open './presets/cmaf_audio_aac_64kbps.json'"," at Object.openSync (fs.js:462:3)"," at Object.readFileSync (fs.js:364:35)"," at /var/task/lambda-configure/media-convert.js:48:32"," at new Promise (<anonymous>)"," at createPreset (/var/task/lambda-configure/media-convert.js:47:10)"," at /var/task/lambda-configure/media-convert.js:88:47"," at Array.forEach (<anonymous>)"," at /var/task/lambda-configure/media-convert.js:88:13"," at new Promise (<anonymous>)"," at Object.createPresets [as MediaConvertPresets] (/var/task/lambda-configure/media-convert.js:84:10)"]})
Error: Error creating IAM policy default-marsha-migrate-lambda-invoke-policy: EntityAlreadyExists: A policy called default-marsha-migrate-lambda-invoke-policy already exists. Duplicate names are not allowed.
status code: 409, request id: 1953a688-e514-4d60-a0ea-e5082a9fa65a
I seem to have forgotten some strategies to delete I raise the make sorry
scwall
commented
3 years ago after the removal of all the strategies and groups, I started again and I had only 2 errors (I'm close to the goal)
Error: Lambda function (default-marsha-configure) returned error: ({"errorType":"Error","errorMessage":"ENOENT: no such file or directory, open './presets/cmaf_audio_aac_64kbps.json'","trace":["Error: ENOENT: no such file or directory, open './presets/cmaf_audio_aac_64kbps.json'"," at Object.openSync (fs.js:462:3)"," at Object.readFileSync (fs.js:364:35)"," at /var/task/lambda-configure/media-convert.js:48:32"," at new Promise (<anonymous>)"," at createPreset (/var/task/lambda-configure/media-convert.js:47:10)"," at /var/task/lambda-configure/media-convert.js:88:47"," at Array.forEach (<anonymous>)"," at /var/task/lambda-configure/media-convert.js:88:13"," at new Promise (<anonymous>)"," at Object.createPresets [as MediaConvertPresets] (/var/task/lambda-configure/media-convert.js:84:10)"]})
Error: Error creating IAM User default-marsha: EntityAlreadyExists: User with name default-marsha already exists.
status code: 409, request id: c94a0d9f-60f9-4229-a5eb-bbfb2c723029
Then I still have two errors, it seems that there is a missing file for the audio configuration and it tries to create the user default-marsha.
Then I still have two errors, it seems that a file for the audio configuration is missing and it tries to create the user default-marsha. I deleted the user default-marsha, then I deleted the policies and groups again. But when I restarted it tells me
I guess it's because I deleted the user default-marsha
Error: AccessDeniedException:
status code: 403, request id: 6446f08e-81d0-49ab-810f-9fbfdf9ab7a5
make: *** [Makefile:7: apply] Error 1I guess it's because I deleted the user
lunika
commented
3 years ago src/aws/lambda-configure/presets/cmaf_audio_aac_64kbps.json ? Or does it have inaccessible right ?default-marsha was already existing before applying the planscwall
commented
3 years ago Hello, We tested on friday and we reset all the amazon plan removing all the accesses, finally the big cleanup. After a re-installation we still have an error:
data.aws_lambda_invocation.configure_lambda_presets: Still reading... [7m30s elapsed]
data.aws_lambda_invocation.invoke_migration: Still reading... [6m10s elapsed]
data.aws_lambda_invocation.configure_lambda_presets: Still reading... [7m40s elapsed]
data.aws_lambda_invocation.invoke_migration: Still reading... [6m20s elapsed]
data.aws_lambda_invocation.configure_lambda_presets: Still reading... [7m50s elapsed]
data.aws_lambda_invocation.invoke_migration: Still reading... [6m30s elapsed]
data.aws_lambda_invocation.configure_lambda_presets: Still reading... [8m0s elapsed]
data.aws_lambda_invocation.invoke_migration: Still reading... [6m40s elapsed]
data.aws_lambda_invocation.configure_lambda_presets: Still reading... [8m10s elapsed]
data.aws_lambda_invocation.invoke_migration: Still reading... [6m50s elapsed]
data.aws_lambda_invocation.configure_lambda_presets: Still reading... [8m20s elapsed]
data.aws_lambda_invocation.invoke_migration: Still reading... [7m0s elapsed]
data.aws_lambda_invocation.configure_lambda_presets: Still reading... [8m30s elapsed]
data.aws_lambda_invocation.invoke_migration: Still reading... [7m10s elapsed]
data.aws_lambda_invocation.invoke_migration: Read complete after 7m11s [id=default-marsha-migrate_$LATEST_63ba95b888672b0c57a663ed581d4db5]
Error: Lambda function (default-marsha-configure) returned error: ({"errorType":"Error","errorMessage":"ENOENT: no such file or directory, open './presets/cmaf_audio_aac_64kbps.json'","trace":["Error: ENOENT: no such file or directory, open './presets/cmaf_audio_aac_64kbps.json'"," at Object.openSync (fs.js:462:3)"," at Object.readFileSync (fs.js:364:35)"," at /var/task/lambda-configure/media-convert.js:48:32"," at new Promise (<anonymous>)"," at createPreset (/var/task/lambda-configure/media-convert.js:47:10)"," at /var/task/lambda-configure/media-convert.js:88:47"," at Array.forEach (<anonymous>)"," at /var/task/lambda-configure/media-convert.js:88:13"," at new Promise (<anonymous>)"," at Object.createPresets [as MediaConvertPresets] (/var/task/lambda-configure/media-convert.js:84:10)"]})
make: *** [Makefile:7: apply] Error 1However, the file is present in the marsha project folders.
lunika
commented
3 years ago This is an issue in the lambda-configure function. Here is the commit to solve the issue: https://github.com/openfun/marsha/pull/820/commits/4f56d0a455c3eb120cb76262ea8df55b9b7c2296
lunika
commented
3 years ago Is it ok for you with this last commit ?
lunika
commented
3 years ago done in #834 and #820
Hello I am currently immersed in the deployment of your project,
I noticed that there are errors in the readme.md the make state-create command, no longer exists in any make, make deploy command only exists until v3.12.0 I also noticed that in the aws\state files. tf create_state_bucket\s3.tf and shared_resources\state.tf the bucket is referenced bucket = "marsha-terraform" and there is no variable to specify the name of the bucket in the environment file, I had to modify the files manually to designate the bucket example that could be proposed :
Readme :
Create the shared state bucket where will keep all the information on your deployments so different developers/machines/CI processes can interact with them:Terraform
$ make state-create Initialize your config:Terraform
$ make init Build the lambdas (using ) and automatically configure the infrastructure (this will start incurring billing on ):yarnAWS
$ make deploy
Am I misleading the operation? Thank you in advance 😄