In rdev environments, due to the shape of the IAM policy, we can only upload files to env-rdev-cellxgene-dataset-submissions/*. This could become a problem if we run functional tests on an rdev environment that include the submission lambda. It may also cause other rdev environments to try processing a submission that was not intended for their environment. There is no easy fix at the moment as it would require writing a separate policy for the rdev environment to allow a wild card. The new IAM policy resource would like something like this:
This would allow for the rdev environment name to be present. Some additional tweak to the submission lambda to parse out the rdev environment for the key name.
The post_s3_credentials will also need to be updated to parse the submission bucket name to only include the bucket name and add the rdev environment to the upload path.
Expected behavior
Datasets uploaded to the rdev submission bucket should only be process by that rdev environment.
Describe the bug
In rdev environments, due to the shape of the IAM policy, we can only upload files to
env-rdev-cellxgene-dataset-submissions/*
. This could become a problem if we run functional tests on an rdev environment that include the submission lambda. It may also cause other rdev environments to try processing a submission that was not intended for their environment. There is no easy fix at the moment as it would require writing a separate policy for the rdev environment to allow a wild card. The new IAM policy resource would like something like this:This would allow for the rdev environment name to be present. Some additional tweak to the submission lambda to parse out the rdev environment for the key name.
The
post_s3_credentials
will also need to be updated to parse the submission bucket name to only include the bucket name and add the rdev environment to the upload path.Expected behavior
Datasets uploaded to the rdev submission bucket should only be process by that rdev environment.