SSM connector docs should explain the S3 part

mdavis-xyz commented 1 year ago

Summary

The SSM connector docs don't mention S3 up the top.

They only mention it in the details of the arguments, which is a bit unclear for someone completely new to this.

In the "Requirements" section, it should say

that you need to have already created an S3 bucket.
that the target and controller both need network connectivity to S3
why a bucket is required, even if you're not running any copy commands. (One sentence explanation is probably fine.)
which IAM permissions are required on the target (e.g. s3:GetObject, or s3:GetObjectVersion, etc, or also ListBucket?)
which IAM permissions are required on the controller (s3:PutObject, s3:DeleteObject. Anything else? e.g. presigned URLs?)
which prefix within S3 the objects are saved to
whether the files in S3 are deleted when done.
whether the files in S3 are deleted if the general Ansible setting keep_remote_files=True.

Issue Type

Documentation Report

Component Name

community.aws.aws_ssm connection

Ansible Version

ansible [core 2.13.5]
  config file = /Users/matthew/Documents/mms/new-repo/ansible.cfg
  configured module search path = ['/Users/matthew/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/matthew/.pyenv/versions/3.10.0/lib/python3.10/site-packages/ansible
  ansible collection location = /Users/matthew/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/matthew/.pyenv/versions/3.10.0/bin/ansible
  python version = 3.10.0 (default, Nov 12 2021, 11:20:43) [Clang 12.0.5 (clang-1205.0.22.11)]
  jinja version = 3.1.2
  libyaml = False

Collection Versions

$ ansible-galaxy collection list
# /Users/matthew/.pyenv/versions/3.10.0/lib/python3.10/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    3.4.0  
ansible.netcommon             3.1.1  
ansible.posix                 1.4.0  
ansible.utils                 2.6.1  
ansible.windows               1.11.1 
arista.eos                    5.0.1  
awx.awx                       21.5.0 
azure.azcollection            1.13.0 
check_point.mgmt              2.3.0  
chocolatey.chocolatey         1.3.0  
cisco.aci                     2.2.0  
cisco.asa                     3.1.0  
cisco.dnac                    6.6.0  
cisco.intersight              1.0.19 
cisco.ios                     3.3.1  
cisco.iosxr                   3.3.1  
cisco.ise                     2.5.3  
cisco.meraki                  2.11.0 
cisco.mso                     2.0.0  
cisco.nso                     1.0.3  
cisco.nxos                    3.1.1  
cisco.ucs                     1.8.0  
cloud.common                  2.1.2  
cloudscale_ch.cloud           2.2.2  
community.aws                 3.5.0  
community.azure               1.1.0  
community.ciscosmb            1.0.5  
community.crypto              2.5.0  
community.digitalocean        1.21.0 
community.dns                 2.3.2  
community.docker              2.7.1  
community.fortios             1.0.0  
community.general             5.6.0  
community.google              1.0.0  
community.grafana             1.5.2  
community.hashi_vault         3.2.0  
community.hrobot              1.5.2  
community.libvirt             1.2.0  
community.mongodb             1.4.2  
community.mysql               3.5.1  
community.network             4.0.1  
community.okd                 2.2.0  
community.postgresql          2.2.0  
community.proxysql            1.4.0  
community.rabbitmq            1.2.2  
community.routeros            2.3.0  
community.sap                 1.0.0  
community.sap_libs            1.3.0  
community.skydive             1.0.0  
community.sops                1.4.0  
community.vmware              2.9.1  
community.windows             1.11.0 
community.zabbix              1.8.0  
containers.podman             1.9.4  
cyberark.conjur               1.2.0  
cyberark.pas                  1.0.14 
dellemc.enterprise_sonic      1.1.2  
dellemc.openmanage            5.5.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.19.0 
fortinet.fortimanager         2.1.5  
fortinet.fortios              2.1.7  
frr.frr                       2.0.0  
gluster.gluster               1.0.2  
google.cloud                  1.0.2  
hetzner.hcloud                1.8.2  
hpe.nimble                    1.1.4  
ibm.qradar                    2.1.0  
ibm.spectrum_virtualize       1.9.0  
infinidat.infinibox           1.3.3  
infoblox.nios_modules         1.3.0  
inspur.ispim                  1.0.1  
inspur.sm                     2.0.0  
junipernetworks.junos         3.1.0  
kubernetes.core               2.3.2  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.19.0
netapp.elementsw              21.7.0 
netapp.ontap                  21.23.0
netapp.storagegrid            21.11.0
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.3.1  
netbox.netbox                 3.7.1  
ngine_io.cloudstack           2.2.4  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.2  
openstack.cloud               1.9.1  
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   2.2.3  
purestorage.flasharray        1.13.0 
purestorage.flashblade        1.10.0 
purestorage.fusion            1.1.0  
sensu.sensu_go                1.13.1 
servicenow.servicenow         1.0.6  
splunk.es                     2.1.0  
t_systems_mms.icinga_director 1.31.0 
theforeman.foreman            3.6.0  
vmware.vmware_rest            2.2.0  
vultr.cloud                   1.1.0  
vyos.vyos                     3.0.1  
wti.remote                    1.0.4

Configuration

$ ansible-config dump --only-changed
ANY_ERRORS_FATAL(/Users/matthew/Documents/mms/new-repo/ansible.cfg) = True
DEFAULT_KEEP_REMOTE_FILES(env: ANSIBLE_KEEP_REMOTE_FILES) = False
DEFAULT_STDOUT_CALLBACK(/Users/matthew/Documents/mms/new-repo/ansible.cfg) = yaml
INVENTORY_UNPARSED_IS_FAILED(/Users/matthew/Documents/mms/new-repo/ansible.cfg) = True
LOCALHOST_WARNING(/Users/matthew/Documents/mms/new-repo/ansible.cfg) = False

OS / Environment

Mac OS

Additional Information

No response

Code of Conduct

[X] I agree to follow the Ansible Code of Conduct

markuman commented 1 year ago

@mdavis-xyz looks like you're more familar with ssm connections.
Are you willing to provide a PR that improves the documentation?

mdavis-xyz commented 1 year ago

I need confirmation of the answers. e.g. I'm not sure what permissions are required for the controller. especially for presigned URLs.

Also, what's the right way to put hyperlinks in the docs? e.g. keep_remote_files=True? I can put in that absolute hyperlink, but I assume we'd want to keep the links within each particular version of the docs?

Q: Why a bucket is required, even if you're not running any copy commands. (One sentence explanation is probably fine.):

A: Ansible is to designed to not require anything (except Python) to be installed on the target. For each Ansible module, Ansible copies a python script to the target, and then executes it. This is true for all modules, not just the file copying ones like copy. It is possible to send files directly over SSM, however that is slow compared to using S3. That is, the controller uploads files to S3, and then sends a shell command to the target, telling it to download the file from S3.

Q: Which IAM permissions are required on the target (e.g. s3:GetObject, or s3:GetObjectVersion, etc, or also ListBucket?) A: No S3 IAM permissions are required on the target. To simplify IAM permissions and reduce dependency requirements, the controller generates a pre-signed URL for each file, and then tells the target to run curl https://....

Q: which IAM permissions are required on the controller (s3:PutObject, s3:DeleteObject. Anything else? e.g. presigned URLs?) A: I'm not sure

Q: which prefix within S3 the objects are saved to A: The file /path/to/something.txt For EC2 instance i-123 will be saved at s3://bucket/i-123//path/to/something.txt.

Q: whether the files in S3 are deleted when done. A: yes it does

A: whether the files in S3 are deleted if the general Ansible setting keep_remote_files=True. Q: That setting is ignored for files in S3.

mdavis-xyz commented 1 year ago

One other reason for S3 is that if you send files directly over SSM (e.g. echo blah | base64 -d > file), the contents will be visible persistently in .bash_history, and perhaps in SSM execution history. For some files that might be a security risk. (Just a guess)

simon97k commented 1 year ago

I was struggling with the S3 permissions as well due to missing documentary. Finally I found out that the Ansible host as the target need these actions allowed: s3:GetObject, s3:PutObject, s3:ListBucket, s3:DeleteObject and s3:GetBucketLocation. I did this using a bucket policy.

A short documentation would be helpful, thank you!

Edit: Update required actions

ansible-collections / community.aws