4dn-dcic / tibanna

Tibanna helps you run your genomic pipelines on Amazon cloud (AWS). It is used by the 4DN DCIC (4D Nucleome Data Coordination and Integration Center) to process data. Tibanna supports CWL/WDL (w/ docker), Snakemake (w/ conda) and custom Docker/shell command.
MIT License
70 stars 28 forks source link

KMS Authentication Error in Lambda When Running Tibanna #402

Closed yuji-yoshida-craif closed 6 months ago

yuji-yoshida-craif commented 6 months ago

After executing the command: tibanna deploy_unicorn -g tibanna_group -b bucket I proceeded to run: tibanna run_workflow --input-json=tibanna_input.json However, I encountered an error in the first Lambda function during the Step Function execution:

{
  "cause": "Lambda was unable to decrypt the environment variables because KMS access was denied. Please check the function's KMS key settings. KMS Exception: AccessDeniedExceptionKMS Message: User: arn:aws:sts::aws-account:assumed-role/tibanna_a_run_task_awsem/run_task_awsem_tibanna_a is not authorized to perform: kms:Decrypt on resource: arn:aws:kms:resion:aws-acount:key/key_ID because no resource-based policy allows the kms:Decrypt action (Service: Lambda, Status Code: 502, Request ID: ID)",
  "error": "Lambda.KMSAccessDeniedException"
}

Could you help identify the cause and suggest a solution?

Here is the input.json I used:

{
  "args": {
    "language": "snakemake",
    "snakemake_directory_url": "s3://bucket",
    "snakemake_main_filename": "Snakefile",
    "container_image": "snakemake/snakemake",
    "command": "snakemake --use-conda all",
    "output_S3_bucket": "bucket",
    "input_files": {
      "fastq_files": {
        "bucket_name": "bucket",
        "object_key": "fastq/*.fastq.gz",
        "mount": true
      }
    }
  },
  "config": {
    "log_bucket": "log-bucket",
    "instance_type": ["t3.medium", "t3.small"],
    "mem": 8,
    "mem_as_is": false,
    "cpu": 2,
    "ebs_size": 20,
    "ebs_size_as_is": false,
    "EBS_optimized": true,
    "root_ebs_size": 8,
    "shutdown_min": "now",
    "password": "",
    "key_name": "",
    "ebs_iops": null,
    "ebs_throughput": null,
    "ebs_type": "gp3",
    "spot_instance": false,
    "spot_duration": null,
    "behavior_on_capacity_limit": "fail",
    "availability_zone": "",
    "security_group": "",
    "subnet": ""
  }
}
alexander-veit commented 6 months ago

It seems you are using server side encryption. Try setting the following environment variable:

export S3_ENCRYPT_KEY_ID=<YOUR_KEY_ID>

After that you need to redeploy Tibanna.