Open anneadb opened 1 month ago
Voting for Prioritization
Volunteering to Work on This Issue
Hey @anneadb 👋 Thank you for taking the time to raise this! Are you able to supply debug logs (redacted as needed)? While I would expect space_settings
to be required (it mentions that in the [documentation]()), the error around lifecycle_config_arns
being unexpected is odd to me, and something I wasn't able to reproduce with the following stripped-down configuration and terraform validate
:
variable "test" {}
resource "aws_sagemaker_space" "this" {
domain_id = var.test
space_name = "test"
ownership_settings {
owner_user_profile_name = var.test
}
space_settings {
app_type = "JupyterLab"
jupyter_lab_app_settings {
default_resource_spec {
instance_type = "ml.t3.medium"
lifecycle_config_arn = var.test
}
}
}
space_sharing_settings {
sharing_type = "Private"
}
}
Reproduction attempt output:
$ terraform --version
Terraform v1.9.2
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.58.0
$ terraform validate
Success! The configuration is valid.
Hi @justinretzolk,
In general I find it slightly confusing that e.g. space_settings is required but then all options within space_settings are listed as optional. I would assume that an empty {} would then be allowed but apparently it is not? But maybe this is a general Terraform convention that I have not learned yet :D
With just space_settings {}
:
validate -> Success! The configuration is valid.
apply -> │ Error: creating SageMaker Space: ValidationException: AppType [null] is not supported for private spaces. Use [CodeEditor, JupyterLab] instead.
Maybe these requirements could be made a bit clearer in the documentation.
Regarding lifecycle_config_arns
: according to https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sagemaker_space#jupyter-lab-app-settings, lifecycle_config_arns
is a valid input and default_resource_spec
is optional.
However with the following
resource "aws_sagemaker_studio_lifecycle_config" "hello" {
studio_lifecycle_config_name = "hello"
studio_lifecycle_config_app_type = "JupyterLab"
studio_lifecycle_config_content = base64encode(
"echo hello"
)
}
...
resource "aws_sagemaker_space" "this" {
domain_id = aws_sagemaker_domain.this[0].id
space_name = "test"
ownership_settings {
owner_user_profile_name = aws_sagemaker_user_profile.this[0].user_profile_name
}
space_settings {
jupyter_lab_app_settings {
lifecycle_config_arns = [aws_sagemaker_studio_lifecycle_config.hello.arn]
}
}
space_sharing_settings {
sharing_type = "Private"
}
}
I get this output:
Error: Insufficient default_resource_spec blocks
│
│ on main_sagemaker.tf line 94, in resource "aws_sagemaker_space" "this":
│ 94: jupyter_lab_app_settings {
│
│ At least 1 "default_resource_spec" blocks are required.
╵
╷
│ Error: Unsupported argument
│
│ on main_sagemaker.tf line 95, in resource "aws_sagemaker_space" "this":
│ 95: lifecycle_config_arns = [aws_sagemaker_studio_lifecycle_config.hello.arn]
│
│ An argument named "lifecycle_config_arns" is not expected here.
Full debug output (I removed all unrelated resources from my files before generating this so I hope it is not too cluttered):
2024-07-17T15:00:45.682+0200 [INFO] Terraform version: 1.9.0
2024-07-17T15:00:45.682+0200 [DEBUG] using github.com/hashicorp/go-tfe v1.51.0
2024-07-17T15:00:45.682+0200 [DEBUG] using github.com/hashicorp/hcl/v2 v2.20.0
2024-07-17T15:00:45.682+0200 [DEBUG] using github.com/hashicorp/terraform-svchost v0.1.1
2024-07-17T15:00:45.682+0200 [DEBUG] using github.com/zclconf/go-cty v1.14.4
2024-07-17T15:00:45.682+0200 [INFO] Go runtime version: go1.22.4
2024-07-17T15:00:45.682+0200 [INFO] CLI args: []string{"/opt/homebrew/Cellar/tfenv/3.0.0/versions/1.9.0/terraform", "-chdir=infrastructure", "validate"}
2024-07-17T15:00:45.682+0200 [DEBUG] Attempting to open CLI config file: /Users/***/.terraformrc
2024-07-17T15:00:45.682+0200 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2024-07-17T15:00:45.682+0200 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2024-07-17T15:00:45.682+0200 [DEBUG] ignoring non-existing provider search directory /Users/***/.terraform.d/plugins
2024-07-17T15:00:45.682+0200 [DEBUG] ignoring non-existing provider search directory /Users/***/Library/Application Support/io.terraform/plugins
2024-07-17T15:00:45.682+0200 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2024-07-17T15:00:45.682+0200 [INFO] CLI command args: []string{"validate"}
2024-07-17T15:00:45.948+0200 [DEBUG] checking for provisioner in "."
2024-07-17T15:00:45.948+0200 [DEBUG] checking for provisioner in "/opt/homebrew/Cellar/tfenv/3.0.0/versions/1.9.0"
2024-07-17T15:00:45.948+0200 [DEBUG] created provider logger: level=debug
2024-07-17T15:00:45.948+0200 [INFO] provider: configuring client automatic mTLS
2024-07-17T15:00:45.954+0200 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.58.0/darwin_arm64/terraform-provider-aws_v5.58.0_x5 args=[".terraform/providers/registry.terraform.io/hashicorp/aws/5.58.0/darwin_arm64/terraform-provider-aws_v5.58.0_x5"]
2024-07-17T15:00:45.971+0200 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.58.0/darwin_arm64/terraform-provider-aws_v5.58.0_x5 pid=86800
2024-07-17T15:00:45.971+0200 [DEBUG] provider: waiting for RPC address: plugin=.terraform/providers/registry.terraform.io/hashicorp/aws/5.58.0/darwin_arm64/terraform-provider-aws_v5.58.0_x5
2024-07-17T15:00:46.090+0200 [INFO] provider.terraform-provider-aws_v5.58.0_x5: configuring server automatic mTLS: timestamp="2024-07-17T15:00:46.090+0200"
2024-07-17T15:00:46.096+0200 [DEBUG] provider: using plugin: version=5
2024-07-17T15:00:46.096+0200 [DEBUG] provider.terraform-provider-aws_v5.58.0_x5: plugin address: address=/var/folders/sp/5br45gn159d_pwcqfqrnvw9d6rng0h/T/plugin1386450190 network=unix timestamp="2024-07-17T15:00:46.096+0200"
2024-07-17T15:00:46.488+0200 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2024-07-17T15:00:46.491+0200 [INFO] provider: plugin process exited: plugin=.terraform/providers/registry.terraform.io/hashicorp/aws/5.58.0/darwin_arm64/terraform-provider-aws_v5.58.0_x5 id=86800
2024-07-17T15:00:46.491+0200 [DEBUG] provider: plugin exited
2024-07-17T15:00:46.491+0200 [DEBUG] Building and walking validate graph
2024-07-17T15:00:46.491+0200 [DEBUG] ProviderTransformer: "aws_sagemaker_studio_lifecycle_config.hello" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2024-07-17T15:00:46.491+0200 [DEBUG] ProviderTransformer: "aws_iam_role.analytics" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2024-07-17T15:00:46.491+0200 [DEBUG] ProviderTransformer: "data.aws_ssm_parameter.kms_key_arn" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2024-07-17T15:00:46.491+0200 [DEBUG] ProviderTransformer: "aws_sagemaker_space.this" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2024-07-17T15:00:46.491+0200 [DEBUG] ProviderTransformer: "aws_security_group.this" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2024-07-17T15:00:46.491+0200 [DEBUG] ProviderTransformer: "aws_sagemaker_domain.this" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2024-07-17T15:00:46.491+0200 [DEBUG] ProviderTransformer: "data.aws_ssm_parameter.vpc_id" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2024-07-17T15:00:46.491+0200 [DEBUG] ProviderTransformer: "aws_sagemaker_user_profile.this" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2024-07-17T15:00:46.491+0200 [DEBUG] ProviderTransformer: "data.aws_vpc.this" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2024-07-17T15:00:46.491+0200 [DEBUG] ProviderTransformer: "data.aws_subnets.this" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2024-07-17T15:00:46.491+0200 [DEBUG] ProviderTransformer: "data.aws_iam_policy_document.analytics_assume_role" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "var.workload (validation)" references: []
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "aws_security_group.this" references: [data.aws_ssm_parameter.vpc_id data.aws_vpc.this]
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "aws_sagemaker_domain.this" references: [var.create_sagemaker_studio data.aws_subnets.this data.aws_ssm_parameter.vpc_id local.resource_name_prefix (expand) data.aws_ssm_parameter.kms_key_arn aws_iam_role.analytics aws_security_group.this aws_security_group.this aws_iam_role.analytics aws_security_group.this]
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "data.aws_subnets.this" references: []
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "data.aws_iam_policy_document.analytics_assume_role" references: [var.create_analytics_resources]
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "var.workload" references: []
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "aws_sagemaker_studio_lifecycle_config.hello" references: []
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "var.aws_account_id" references: []
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/aws\"]" references: [var.aws_region var.aws_account_id]
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "aws_iam_role.analytics" references: [var.create_analytics_resources local.resource_name_prefix (expand) data.aws_iam_policy_document.analytics_assume_role]
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "data.aws_ssm_parameter.kms_key_arn" references: []
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "data.aws_ssm_parameter.vpc_id" references: []
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "var.create_sagemaker_studio" references: []
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "local.resource_name_prefix (expand)" references: [var.workload (validation) var.workload]
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "aws_sagemaker_user_profile.this" references: [var.create_sagemaker_studio aws_sagemaker_domain.this]
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "aws_sagemaker_space.this" references: [aws_sagemaker_domain.this aws_sagemaker_user_profile.this]
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "data.aws_vpc.this" references: [data.aws_ssm_parameter.vpc_id]
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "var.aws_region" references: []
2024-07-17T15:00:46.492+0200 [DEBUG] ReferenceTransformer: "var.create_analytics_resources" references: []
2024-07-17T15:00:46.493+0200 [DEBUG] Starting graph walk: walkValidate
2024-07-17T15:00:46.493+0200 [DEBUG] created provider logger: level=debug
2024-07-17T15:00:46.493+0200 [INFO] provider: configuring client automatic mTLS
2024-07-17T15:00:46.496+0200 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.58.0/darwin_arm64/terraform-provider-aws_v5.58.0_x5 args=[".terraform/providers/registry.terraform.io/hashicorp/aws/5.58.0/darwin_arm64/terraform-provider-aws_v5.58.0_x5"]
2024-07-17T15:00:46.518+0200 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.58.0/darwin_arm64/terraform-provider-aws_v5.58.0_x5 pid=86827
2024-07-17T15:00:46.518+0200 [DEBUG] provider: waiting for RPC address: plugin=.terraform/providers/registry.terraform.io/hashicorp/aws/5.58.0/darwin_arm64/terraform-provider-aws_v5.58.0_x5
2024-07-17T15:00:46.640+0200 [INFO] provider.terraform-provider-aws_v5.58.0_x5: configuring server automatic mTLS: timestamp="2024-07-17T15:00:46.640+0200"
2024-07-17T15:00:46.646+0200 [DEBUG] provider.terraform-provider-aws_v5.58.0_x5: plugin address: address=/var/folders/sp/5br45gn159d_pwcqfqrnvw9d6rng0h/T/plugin3782417008 network=unix timestamp="2024-07-17T15:00:46.646+0200"
2024-07-17T15:00:46.646+0200 [DEBUG] provider: using plugin: version=5
2024-07-17T15:00:46.718+0200 [ERROR] vertex "aws_sagemaker_space.this" error: Unsupported argument
2024-07-17T15:00:46.718+0200 [ERROR] vertex "aws_sagemaker_space.this" error: Insufficient default_resource_spec blocks
╷
│ Error: Insufficient default_resource_spec blocks
│
│ on main_sagemaker.tf line 94, in resource "aws_sagemaker_space" "this":
│ 94: jupyter_lab_app_settings {
│
│ At least 1 "default_resource_spec" blocks are required.
╵
╷
│ Error: Unsupported argument
│
│ on main_sagemaker.tf line 95, in resource "aws_sagemaker_space" "this":
│ 95: lifecycle_config_arns = [aws_sagemaker_studio_lifecycle_config.hello.arn]
│
│ An argument named "lifecycle_config_arns" is not expected here.
╵
2024-07-17T15:00:46.720+0200 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2024-07-17T15:00:46.722+0200 [INFO] provider: plugin process exited: plugin=.terraform/providers/registry.terraform.io/hashicorp/aws/5.58.0/darwin_arm64/terraform-provider-aws_v5.58.0_x5 id=86827
2024-07-17T15:00:46.722+0200 [DEBUG] provider: plugin exited
Thanks for the additional info @anneadb! I've opened a pull request to fix some of the documentation based on that -- on reviewing the resource schema, I found that the documentation was wrong in both of those places. I also tried to make the documentation a bit more clear by updating it so that optional are marked as such, but if any of the block's arguments are required, they're marked as such.
Hi @justinretzolk ,
thanks for the update! Have you had any luck figuring out why "app_type" is at first required but then cannot be "updated" (although there is no change in Terraform config)?
Any further updates on this?
I am running into the same app_type
error on provider version 5.60.0
I came across the same problem. now it is resolved now.
The missing part is default_space_settings block in default_user_settings
default_space_settings{
execution_role=module.sagemaker_domain_execution_role.default_execution_role
}
The complete code is below- ---------------------------sagemaker.tf----------------------------- data "aws_caller_identity" "current" {}
data "aws_region" "current" {}
resource "aws_kms_key" "sagemaker_efs_kms_key" { description = "KMS key used to encrypt SageMaker Studio EFS volume" enable_key_rotation = true }
resource "aws_kms_key_policy" "example" { key_id = aws_kms_key.sagemaker_efs_kms_key.id policy = jsonencode({ Id = "example" Statement = [ { Action = "kms:*" Effect = "Allow" Principal = { AWS = [data.aws_caller_identity.current.account_id] }
Resource = "*"
Sid = "Enable IAM User Permissions"
},
]
Version = "2012-10-17"
}) }
module "sagemaker_domain_execution_role" { source = "./submodules/iam" kms_arn = aws_kms_key.sagemaker_efs_kms_key.arn }
module "sagemaker_domain_vpc" { source = "./submodules/vpc" cidr_block = local.vpc.cidr_block private_subnet_cidrs = local.vpc.private_subnet_cidrs azs = local.vpc.availability_zones }
module "auto_shutdown_s3_upload" { source = "./submodules/s3_upload" kms_arn = aws_kms_key.sagemaker_efs_kms_key.arn }
resource "aws_sagemaker_studio_lifecycle_config" "auto_shutdown" { studio_lifecycle_config_name = "auto-shutdown" studio_lifecycle_config_app_type = "JupyterServer" studio_lifecycle_config_content = base64encode(templatefile("${path.module}/assets/auto_shutdown_template/autoshutdown-script.sh", { tar_file_bucket = module.auto_shutdown_s3_upload.tar_file_bucket, tar_file_id = module.auto_shutdown_s3_upload.tar_file_id })) }
resource "aws_sagemaker_domain" "sagemaker_domain" { domain_name = var.domain_name auth_mode = var.auth_mode vpc_id = module.sagemaker_domain_vpc.vpc_id subnet_ids = module.sagemaker_domain_vpc.private_subnet_ids
default_user_settings { execution_role = module.sagemaker_domain_execution_role.default_execution_role jupyter_server_app_settings { default_resource_spec { lifecycle_config_arn = aws_sagemaker_studio_lifecycle_config.auto_shutdown.arn sagemaker_image_arn = local.sagemaker_image_arn } lifecycle_config_arns = [aws_sagemaker_studio_lifecycle_config.auto_shutdown.arn] }
canvas_app_settings {
time_series_forecasting_settings {
status = "ENABLED"
}
}
space_storage_settings{
default_ebs_storage_settings {
default_ebs_volume_size_in_gb =100
maximum_ebs_volume_size_in_gb =100
}
}
}
default_space_settings {
execution_role=module.sagemaker_domain_execution_role.default_execution_role
}
domain_settings { security_group_ids = [module.sagemaker_domain_vpc.security_group_id] }
kms_key_id = aws_kms_key.sagemaker_efs_kms_key.arn
app_network_access_type = var.app_network_access_type
retention_policy { home_efs_file_system = var.efs_retention_policy } }
resource "aws_sagemaker_user_profile" "default_user" { domain_id = aws_sagemaker_domain.sagemaker_domain.id user_profile_name = "defaultuser"
user_settings { execution_role = module.sagemaker_domain_execution_role.default_execution_role security_groups = [module.sagemaker_domain_vpc.security_group_id] } }
resource "aws_sagemaker_space" "sagemaker_space" { domain_id = aws_sagemaker_domain.sagemaker_domain.id space_name = "poc"
ownership_settings { owner_user_profile_name = aws_sagemaker_user_profile.default_user.user_profile_name }
space_settings { app_type = "JupyterLab"
jupyter_lab_app_settings {
code_repository {
repository_url = "xxx"
}
default_resource_spec {
instance_type = "ml.t3.medium"
//Refer Images list - https://docs.aws.amazon.com/sagemaker/latest/dg/notebooks-available-images.html
sagemaker_image_arn = "arn:aws:sagemaker:eu-west-2:021081402939:image/sagemaker-distribution-cpu"
sagemaker_image_version_alias = "1.9.0"
}
}
}
space_sharing_settings { sharing_type = "Shared" } }
I hope, It will help you. Thank you!
Hi @532629, I already included "default_space_settings" in my "aws_sagemaker_domain" so I am surprised that this is working for you now. Maybe something else changed? I will try to verify again in the next few days.
Please ensure that you should pass correct execution_role and security group with in it.
default_space_settings {
execution_role=module.sagemaker_domain_execution_role.default_execution_role
//The security groups for the Amazon Virtual Private Cloud that the space uses for communication. For example, internet access to EC2 instance, which is created into space.
security_groups= [module.sagemaker_domain_vpc.security_group_id]
}
Good luck!
Terraform Core Version
1.9.0
AWS Provider Version
5.58.0
Affected Resource(s)
Expected Behavior
After initially creating a space, I would like to be able to modify it.
Actual Behavior
When I initially create a space it requires the "app_type" in space settings. However, when I later on try to modify the space, e.g. by adding a lifecycle configuration I get an error stating that "app_type" cannot be updated. This happens although the app type did not change.
Error when trying to set up space without space_settings:
Error when trying to apply other changes to the space:
Relevant Error/Panic Output Snippet
No response
Terraform Configuration Files
Steps to Reproduce
Debug Output
No response
Panic Output
No response
Important Factoids
I also get another error when trying to add lifecycle_config_arns:
Unexpected attribute: An attribute named "lifecycle_config_arns" is not expected here
. However, this parameter is listed in the documentation.I could not select the lifecycle configuration in the UI when starting the space (it just says "No script"), although it is listed in the AWS console. That's why I am trying to add it in a different way.
Since I am new to using Sagemaker Studio in Terraform, are there any other configuration changes you would recommend to get a lifecycle configuration working for all notebook instances created through jupyter lab?
References
No response
Would you like to implement a fix?
None