copilot managed efs is not accessible between services.

[haorasu]

Hi copilot team,

I was following the developers guide for instantiating a copilot managed EFS here: https://aws.github.io/copilot-cli/docs/developing/storage/

"The use of an access point for each service ensures that no two services can access each other's data unless you specifically intend for them to do so by specifying the full advanced configuration. You can read more in Advanced Use Cases"

I've created 1 test environment, and 2 svc with 1 task each named api and apiv2

Expected behavior was container A writes a file in /var/efs in efs mount point, then container B should be able to see the file. Vice versa should also be true. Some extensive testing from my end with efs:true or setting the uid and gid didn't help with cross service file access.

[iamhopaul123]

Hello @haorasu. Thank you for posting this idea. It totally makes sense to me. Right now we don't have a native support for this scenario. However, it can be easily mitigated using copilot svc override. For the both services, i have the same storage config in the manifest:

storage:
  volumes:
    dbData:
      path: /db-data
      read_only: false
      efs: true

However, for the second one apiv2, I ran copilot svc override and then modified the local cfn.patches.yml file to the following before running copilot svc deploy to deploy the service:

- op: replace
  path: /Resources/AccessPoint/Properties/RootDirectory/Path
  value: /api

Basically we just set the accesspoint to have the same root directory path as the api service (by default we set the accesspoint root dir path to be the service name). This also allows you to configure if you need multiple groups of services (services within the same group can share files).

Please let me know if it doesn't work for you or any questions regarding this workaround.

[haorasu]

Hi @iamhopaul123 thanks for this recommendation. This works for a write once, read many situation. Do you have any recommendations that allows multiple writes from different services?

Right now I have the uid/gid set on new services/containers set to be the same as the very first container/service that was created with the EFS instantiation as a work-around.

[iamhopaul123]

Hello @haorasu sorry I missed this reply.

Right now I have the uid/gid set on new services/containers set to be the same as the very first container/service that was created with the EFS instantiation as a work-around.

That workaround sounds good to me! I'm not sure if you can create an access point with the same uid/gid tho.

Actually after doing some tests it seems like the deployment will fail if you set uid/gid for the new service to be the same as the very first one. I think to allow multiple writes from different services there are two options:

Option 1: reuse the access point

In the very first service you can define an access point with uid/gid set (or let Copilot determines a default one). And in the second service you can reuse that access point:

storage:
  volumes:
    dbData:
      path: /db-data
      read_only: false
      efs:
        id: fs-0a1474d910898cc9c
        auth:
          iam: true
          access_point_id: fsap-0bbcbd7d2c4ec838e

Option 2: override the CreationInfo.Permissions

By default Copilot always sets the permissions to be 755 which basically only allows the owner to write. You can override this value to 775 so that users within the same group can access the files.