search

ansible-collections / community.aws

Ansible Collection for Community AWS
GNU General Public License v3.0
187 stars 395 forks source link

cloudfront_distribution_info summary fails with 'TypeError: CloudFrontFactsServiceManager.describe_cloudfront_property() takes 4 positional arguments but 5 were given' #1915

Open kutzi opened 1 year ago

kutzi commented 1 year ago

Summary

I'm trying to get a summary of all cloudfront distributions. I.e.

community.aws.cloudfront_distribution_info:
    summary: true
    profile: "{{ aws_profile }}"
    region: "{{ aws_region }}

The call fails with:

n exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: CloudFrontFactsServiceManager.describe_cloudfront_property() takes 4 positional arguments but 5 were given
fatal: [DONT]: FAILED! => changed=false 
  module_stderr: |-
    Traceback (most recent call last):
      File "<stdin>", line 107, in <module>
      File "<stdin>", line 99, in _ansiballz_main
      File "<stdin>", line 47, in invoke_module
      File "/usr/lib/python3.10/runpy.py", line 224, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/tmp/ansible_community.aws.cloudfront_distribution_info_payload_tdntauy7/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/community/aws/plugins/modules/cloudfront_distribution_info.py", line 408, in <module>
      File "/tmp/ansible_community.aws.cloudfront_distribution_info_payload_tdntauy7/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/community/aws/plugins/modules/cloudfront_distribution_info.py", line 399, in main
      File "/tmp/ansible_community.aws.cloudfront_distribution_info_payload_tdntauy7/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py", line 158, in summary
      File "/tmp/ansible_community.aws.cloudfront_distribution_info_payload_tdntauy7/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py", line 166, in summary_get_origin_access_identity_list
    TypeError: CloudFrontFactsServiceManager.describe_cloudfront_property() takes 4 positional arguments but 5 were given
  module_stdout: ''
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1

I tried to get the info with a single distribution (i.e. specify distribution: true and distribution_id instead of summary) and that works fine, so I assume that it's not a general issue with my ansible installation

Issue Type

Bug Report

Component Name

community.aws.cloudfront_info

Ansible Version

$ ansible --version
ansible-playbook [core 2.12.7]
  config file = /home/christoph.kutzinski/lab/platform/ansible.cfg
  configured module search path = ['/home/christoph.kutzinski/lab/platform/library', '/home/christoph.kutzinski/lab/platform/infrastructure/kubespray/library']
  ansible python module location = /home/christoph.kutzinski/.ansiblew/5.10.0/a7204d6c/ansible.29711/lib/python3.10/site-packages/ansible
  ansible collection location = /home/christoph.kutzinski/.ansiblew/5.10.0/a7204d6c/ansible/collections
  executable location = /home/christoph.kutzinski/.ansiblew/5.10.0/a7204d6c/ansible/bin/ansible-playbook
  python version = 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
  jinja version = 3.1.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
Collection    Version
------------- -------
amazon.aws    6.3.0  
ansible.utils 2.10.3 
community.aws 6.2.0  
ovirt.ovirt   3.1.2

AWS SDK versions

$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.28.21
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /home/christoph.kutzinski/.ansiblew/5.10.0/a7204d6c/ansible.29711/lib/python3.10/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.31.21
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /home/christoph.kutzinski/.ansiblew/5.10.0/a7204d6c/ansible.29711/lib/python3.10/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: awscli, boto3, s3transfer

Configuration

$ ansible-config dump --only-changed

OS / Environment

Linux Mint

Steps to Reproduce

- name: List distributions
  community.aws.cloudfront_distribution_info:
    summary: true
    profile: "{{ aws_profile }}"
    region: "{{ aws_region }}"
  register: result

- debug: var=result

Expected Results

I expected the module to return a summary of all distributions

Actual Results

TASK [aws/cloudfront : List distributions] **************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: CloudFrontFactsServiceManager.describe_cloudfront_property() takes 4 positional arguments but 5 were given
fatal: [DONT]: FAILED! => changed=false 
  module_stderr: |-
    Traceback (most recent call last):
      File "<stdin>", line 107, in <module>
      File "<stdin>", line 99, in _ansiballz_main
      File "<stdin>", line 47, in invoke_module
      File "/usr/lib/python3.10/runpy.py", line 224, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
        exec(code, run_globals)
      File "/tmp/ansible_community.aws.cloudfront_distribution_info_payload_tdntauy7/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/community/aws/plugins/modules/cloudfront_distribution_info.py", line 408, in <module>
      File "/tmp/ansible_community.aws.cloudfront_distribution_info_payload_tdntauy7/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/community/aws/plugins/modules/cloudfront_distribution_info.py", line 399, in main
      File "/tmp/ansible_community.aws.cloudfront_distribution_info_payload_tdntauy7/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py", line 158, in summary
      File "/tmp/ansible_community.aws.cloudfront_distribution_info_payload_tdntauy7/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py", line 166, in summary_get_origin_access_identity_list
    TypeError: CloudFrontFactsServiceManager.describe_cloudfront_property() takes 4 positional arguments but 5 were given
  module_stdout: ''
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1

Code of Conduct

ansibullbot commented 1 year ago

Files identified in the description: None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

aburgerhulft commented 11 months ago

Having the same issue.

File \"/var/folders/3v/9txpvy690d9fchgwc_v3dp0h0000gn/T/ansible_community.aws.cloudfront_distribution_info_payload_z3n81zfa/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py\", line 166, in summary_get_origin_access_identity_list\nTypeError: CloudFrontFactsServiceManager.describe_cloudfront_property() takes 4 positional arguments but 5 were given\n",

IwkVrr commented 10 months ago

Having the same issue too.

Looks like it's broken since version >6.0.0?

stefanhorning commented 9 months ago

I have the (almost) same error with aws.community version 5.4.0

Error here: CloudFrontFactsServiceManager.paginate_list_cloudfront_property() takes 5 positional arguments but 6 were given

stefanhorning commented 9 months ago

the method is defined over at amazon.aws here https://github.com/ansible-collections/amazon.aws/blob/main/plugins/module_utils/cloudfront_facts.py#L124

Which seems a bit risky to me, as one could have a version of amazon.aws not matching the code being used in the version of community.aws.

stefanhorning commented 9 months ago

In fact a version missmatch was the cause for the issue.

I had the versions

Installing version 5.4.0 for both fixed the issue. But other version combos might also work. It's just down to luck I guess. ;)

stefanhorning commented 9 months ago

The combo 6.3.0 / 6.5.0 also seems to work if you prefer to stay on newer versions.

markuman commented 9 months ago

c.a 5.x.x required a.a >= 5 https://github.com/ansible-collections/community.aws/blob/stable-5/galaxy.yml#L12
Maybe it must also exclude a.a. < 6.

stefanhorning commented 9 months ago

Yes. I guess it's the upper limit that matters here (from community.aws perspective).

IwkVrr commented 9 months ago

I tried with different set of versions but still having the same error

Tried with:

community.aws 7.0.0
amazon.aws 7.1.0
community.aws 7.0.0
amazon.aws 7.0.0
community.aws 6.3.0
amazon.aws 6.5.0

Getting traceback:

File "/tmp/ansible_cloudfront_info_payload_8sg9y1zm/ansible_cloudfront_info_payload.zip/ansible_collections/community/aws/plugins/modules/cloudfront_distribution_info.py", line 408, in <module>
  File "/tmp/ansible_cloudfront_info_payload_8sg9y1zm/ansible_cloudfront_info_payload.zip/ansible_collections/community/aws/plugins/modules/cloudfront_distribution_info.py", line 399, in main
  File "/tmp/ansible_cloudfront_info_payload_8sg9y1zm/ansible_cloudfront_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py", line 158, in summary
  File "/tmp/ansible_cloudfront_info_payload_8sg9y1zm/ansible_cloudfront_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py", line 166, in summary_get_origin_access_identity_list
TypeError: describe_cloudfront_property() takes 4 positional arguments but 5 were given
markuman commented 9 months ago

I cannot reproduce your error.

TypeError: describe_cloudfront_property() takes 4 positional arguments but 5 were given

The function describe_cloudfront_property() was introduced in amazon.aws.6.0.0
https://github.com/ansible-collections/amazon.aws/blob/6.0.0/plugins/module_utils/cloudfront_facts.py#L108
And since it exists, it always accepts >= 4 positional arguments. So this error is an impossible one.

Maybe @abikouo got an idea?

ansible-galaxy collection install 'amazon.aws:<8.0.0' 'community.aws:<8.0.0'

m@zen4:~/git$ ansible-galaxy collection list |grep aws
amazon.aws                     7.1.0  
community.aws                  7.0.0  
amazon.aws                    3.5.0  
community.aws                 3.6.0  
netapp.aws                    21.7.0 
m@zen4:~/git$ ap test.yml 
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] ****************************************************************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************************************************************
ok: [localhost]

TASK [test] *********************************************************************************************************************************************************************************************************
ok: [localhost]

PLAY RECAP **********************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

ansible-galaxy collection install 'amazon.aws:<7.0.0' 'community.aws:<7.0.0'

m@zen4:~/git$ ansible-galaxy collection list |grep aws
amazon.aws                    3.5.0  
community.aws                 3.6.0  
netapp.aws                    21.7.0 
amazon.aws                     6.5.1  
community.aws                  6.4.0  
m@zen4:~/git$ ap test.yml 
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] ****************************************************************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************************************************************
ok: [localhost]

TASK [test] *********************************************************************************************************************************************************************************************************
ok: [localhost]

PLAY RECAP **********************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
m@zen4:~/git$ cat test.yml 
---
- hosts: localhost
  tasks:
    - name: test
      community.aws.cloudfront_distribution_info:
        summary: true
        profile: myaws
        region: eu-central-1
IwkVrr commented 8 months ago

Your ansible-galaxy collection list command output still has community.aws version 3.6.0, maybe that's why you don't get the error?

Tried with your test.yml file and still have the issue:

$ ansible-galaxy collection list

# /home/ec2-user/.ansible/collections/ansible_collections
Collection      Version
--------------- -------
amazon.aws      7.2.0
community.aws   7.1.0
kubernetes.core 3.0.0

$ cat test_summary.yml
---
- hosts: localhost
  tasks:
    - name: test
      community.aws.cloudfront_distribution_info:
        summary: true

$ ansible-playbook test_summary.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] **********************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************
ok: [localhost]

TASK [test] ***************************************************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: describe_cloudfront_property() takes 4 positional arguments but 5 were given
fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"/home/ec2-user/.ansible/tmp/ansible-tmp-1705588826.5443957-3243763-23136583234235/AnsiballZ_cloudfront_distribution_info.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/home/ec2-user/.ansible/tmp/ansible-tmp-1705588826.5443957-3243763-23136583234235/AnsiballZ_cloudfront_distribution_info.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/ec2-user/.ansible/tmp/ansible-tmp-1705588826.5443957-3243763-23136583234235/AnsiballZ_cloudfront_distribution_info.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.cloudfront_distribution_info', init_globals=dict(_module_fqn='ansible_collections.community.aws.plugins.modules.cloudfront_distribution_info', _modlib_path=modlib_path),\n  File \"/usr/lib64/python3.9/runpy.py\", line 225, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib64/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_community.aws.cloudfront_distribution_info_payload_sezyw8_z/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/community/aws/plugins/modules/cloudfront_distribution_info.py\", line 408, in <module>\n  File \"/tmp/ansible_community.aws.cloudfront_distribution_info_payload_sezyw8_z/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/community/aws/plugins/modules/cloudfront_distribution_info.py\", line 399, in main\n  File \"/tmp/ansible_community.aws.cloudfront_distribution_info_payload_sezyw8_z/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py\", line 159, in summary\n  File \"/tmp/ansible_community.aws.cloudfront_distribution_info_payload_sezyw8_z/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py\", line 167, in summary_get_origin_access_identity_list\nTypeError: describe_cloudfront_property() takes 4 positional arguments but 5 were given\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

PLAY RECAP ****************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

And below is the test with list_distributions and list_streaming_distributions that works fine to show that's the setup is fine:

$ cat test_list_distrib.yml
---
- hosts: localhost
  tasks:
    - name: test
      community.aws.cloudfront_distribution_info:
        list_distributions: true
        list_streaming_distributions: true

$ ansible-playbook test_list_distrib.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] **********************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************
ok: [localhost]

TASK [test] ***************************************************************************************************************************************************
ok: [localhost]

PLAY RECAP ****************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
boutetnico commented 7 months ago

This is still an issue.

Playbook

    - name: Retrieve info about CloudFront
      community.aws.cloudfront_distribution_info:
        aws_access_key: "{{ aws_access_key }}"
        aws_secret_key: "{{ aws_secret_key }}"
        region: "{{ aws_region }}"
      register: _cloudfront_distribution_info
      tags: ["route53"]

Error

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: CloudFrontFactsServiceManager.describe_cloudfront_property() takes 4 positional arguments but 5 were given
fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"/home/admin/.ansible/tmp/ansible-tmp-1708424392.9031956-417518-50506265318292/AnsiballZ_cloudfront_distribution_info.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/home/admin/.ansible/tmp/ansible-tmp-1708424392.9031956-417518-50506265318292/AnsiballZ_cloudfront_distribution_info.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/admin/.ansible/tmp/ansible-tmp-1708424392.9031956-417518-50506265318292/AnsiballZ_cloudfront_distribution_info.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.cloudfront_distribution_info', init_globals=dict(_module_fqn='ansible_collections.community.aws.plugins.modules.cloudfront_distribution_info', _modlib_path=modlib_path),\n  File \"<frozen runpy>\", line 226, in run_module\n  File \"<frozen runpy>\", line 98, in _run_module_code\n  File \"<frozen runpy>\", line 88, in _run_code\n  File \"/tmp/ansible_community.aws.cloudfront_distribution_info_payload_1z9jp9dv/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/community/aws/plugins/modules/cloudfront_distribution_info.py\", line 408, in <module>\n  File \"/tmp/ansible_community.aws.cloudfront_distribution_info_payload_1z9jp9dv/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/community/aws/plugins/modules/cloudfront_distribution_info.py\", line 399, in main\n  File \"/tmp/ansible_community.aws.cloudfront_distribution_info_payload_1z9jp9dv/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py\", line 159, in summary\n  File \"/tmp/ansible_community.aws.cloudfront_distribution_info_payload_1z9jp9dv/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py\", line 167, in summary_get_origin_access_identity_list\nTypeError: CloudFrontFactsServiceManager.describe_cloudfront_property() takes 4 positional arguments but 5 were given\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1

Versions

$ ansible --version
ansible [core 2.16.2]
  config file = /home/admin/ansible/ansible.cfg
  configured module search path = ['/home/admin/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/admin/.local/lib/python3.11/site-packages/ansible
  ansible collection location = /home/admin/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True
$ ansible-galaxy collection list | grep aws
amazon.aws                               7.2.0
community.aws                            7.1.0
boutetnico commented 3 months ago

Still happening with these versions:

$ ansible-galaxy collection list | grep aws
amazon.aws                               8.0.0
community.aws                            8.0.0
markuman commented 3 months ago

I don't get it why the error is happen.
And in any case, this is an issue for amazon.aws collection. Because the error happen while calling line 159 https://github.com/ansible-collections/amazon.aws/blob/main/plugins/module_utils/cloudfront_facts.py#L159C9-L159C76

summary_dict.update(self.summary_get_origin_access_identity_list())
  File "/tmp/ansible_community.aws.cloudfront_distribution_info_payload_783zpjan/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py", line 159, in summary
  File "/tmp/ansible_community.aws.cloudfront_distribution_info_payload_783zpjan/ansible_community.aws.cloudfront_distribution_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloudfront_facts.py", line 167, in summary_get_origin_access_identity_list
TypeError: CloudFrontFactsServiceManager.describe_cloudfront_property() takes 4 positional arguments but 5 were given

cc @abikouo @hakbailey @alinabuzachis

davtex commented 5 days ago

I experienced the same issue and found out that it somehow works with "summary: false" and "list_distributions: true"

stefanhorning commented 4 days ago

@markuman yes the error happens in amazon.aws, but community.aws is calling that code wrongly in older versions. As I said in my earlier comment , depending on other projects code is a risky thing as then you have to ensure your dependencies work across all versions of both projects. Hence I see the responsibility on community.aws here.