search

ansible-collections / azure

Development area for Azure Collections
https://galaxy.ansible.com/azure/azcollection
GNU General Public License v3.0
246 stars 330 forks source link

IPv6 address parsed as IPv4 #1561

Open barnabas-kurdi opened 4 months ago

barnabas-kurdi commented 4 months ago
SUMMARY

Local dual IPv6/IPv4 address seems to be parsed as IPv4 when using any module in the Azure collection (by Graph's request adapter). I was able to replicate the issue with multiple modules. I am not sure what the IP is used for in this case in the first place.

ISSUE TYPE
COMPONENT NAME

All modules in the collection seem to be affected.

ANSIBLE VERSION
ansible [core 2.15.11]
  config file = None
  configured module search path = ['/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /runner/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.18 (main, Jan 24 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True
COLLECTION VERSION
Collection         Version
------------------ -------
azure.azcollection 2.3.0
CONFIGURATION
CONFIG_FILE() = None
OS / ENVIRONMENT

The Red Hat Universal Base Image 9 is in use, EE built by ansible-builder, no specific network settings.

STEPS TO REPRODUCE

I tried running the module with and without the proxy, it made no difference. I also tried changing the module arguments, that made no difference whatsoever either. Any module seems to produce a similar error.

---
- name: "Test Azure VM modules"
  hosts: localhost

  tasks:
  - name: "Query VM status"
    azure.azcollection.azure_rm_virtualmachine_info:
      resource_group: <resource group (hidden)>
    environment:
        https_proxy: <corporate proxy (hidden)>
EXPECTED RESULTS

I would except the module to be able to parse the dual address.

ACTUAL RESULTS

In the output below 'BBBB' represents part of the local dual IPv4/IPv6 address (AAAA:BBBB:CCCC:DDDD:xxxx:xxxx:xxxx:xxxx).

module_stdout: ''
module_stderr: |
  Traceback (most recent call last):
    File "/usr/local/lib/python3.9/site-packages/httpx/_urlparse.py", line 346, in normalize_port
      port_as_int = int(port)
  ValueError: invalid literal for int() with base 10: 'BBBB::'

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/runner/.ansible/tmp/ansible-tmp-1715768974.3496563-65-93302956278022/AnsiballZ_azure_rm_virtualmachine_info.py", line 107, in <module>
      _ansiballz_main()
    File "/runner/.ansible/tmp/ansible-tmp-1715768974.3496563-65-93302956278022/AnsiballZ_azure_rm_virtualmachine_info.py", line 99, in _ansiballz_main
      invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
    File "/runner/.ansible/tmp/ansible-tmp-1715768974.3496563-65-93302956278022/AnsiballZ_azure_rm_virtualmachine_info.py", line 47, in invoke_module
      runpy.run_module(mod_name='ansible_collections.azure.azcollection.plugins.modules.azure_rm_virtualmachine_info', init_globals=dict(_module_fqn='ansible_collections.azure.azcollection.plugins.modules.azure_rm_virtualmachine_info', _modlib_path=modlib_path),
    File "/usr/lib64/python3.9/runpy.py", line 225, in run_module
      return _run_module_code(code, init_globals, run_name, mod_spec)
    File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code
      _run_code(code, mod_globals, init_globals,
    File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
      exec(code, run_globals)
    File "/tmp/ansible_azure.azcollection.azure_rm_virtualmachine_info_payload_ln1cx03d/ansible_azure.azcollection.azure_rm_virtualmachine_info_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_virtualmachine_info.py", line 335, in <module>
    File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
    File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
    File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
    File "<frozen zipimport>", line 259, in load_module
    File "/tmp/ansible_azure.azcollection.azure_rm_virtualmachine_info_payload_ln1cx03d/ansible_azure.azcollection.azure_rm_virtualmachine_info_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 290, in <module>
    File "/usr/local/lib/python3.9/site-packages/msgraph/__init__.py", line 5, in <module>
      from .graph_request_adapter import GraphRequestAdapter
    File "/usr/local/lib/python3.9/site-packages/msgraph/graph_request_adapter.py", line 5, in <module>
      from msgraph_core import APIVersion, BaseGraphRequestAdapter, GraphClientFactory
    File "/usr/local/lib/python3.9/site-packages/msgraph_core/__init__.py", line 14, in <module>
      from .base_graph_request_adapter import BaseGraphRequestAdapter
    File "/usr/local/lib/python3.9/site-packages/msgraph_core/base_graph_request_adapter.py", line 9, in <module>
      from kiota_http.httpx_request_adapter import HttpxRequestAdapter
    File "/usr/local/lib/python3.9/site-packages/kiota_http/httpx_request_adapter.py", line 58, in <module>
      class HttpxRequestAdapter(RequestAdapter, Generic[ModelType]):
    File "/usr/local/lib/python3.9/site-packages/kiota_http/httpx_request_adapter.py", line 69, in HttpxRequestAdapter
      http_client: httpx.AsyncClient = KiotaClientFactory.create_with_default_middleware(),
    File "/usr/local/lib/python3.9/site-packages/kiota_http/kiota_client_factory.py", line 61, in create_with_default_middleware
      kiota_async_client = KiotaClientFactory.get_default_client() if client is None else client
    File "/usr/local/lib/python3.9/site-packages/kiota_http/kiota_client_factory.py", line 43, in get_default_client
      return httpx.AsyncClient(timeout=timeout, http2=True)
    File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1442, in __init__
      self._mounts: dict[URLPattern, AsyncBaseTransport | None] = {
    File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1443, in <dictcomp>
      URLPattern(key): None
    File "/usr/local/lib/python3.9/site-packages/httpx/_utils.py", line 370, in __init__
      url = URL(pattern)
    File "/usr/local/lib/python3.9/site-packages/httpx/_urls.py", line 115, in __init__
      self._uri_reference = urlparse(url, **kwargs)
    File "/usr/local/lib/python3.9/site-packages/httpx/_urlparse.py", line 248, in urlparse
      parsed_port: int | None = normalize_port(port, scheme)
    File "/usr/local/lib/python3.9/site-packages/httpx/_urlparse.py", line 348, in normalize_port
      raise InvalidURL(f"Invalid port: {port!r}")
  httpx.InvalidURL: Invalid port: '7ac0::'
exception: |
  Traceback (most recent call last):
    File "/usr/local/lib/python3.9/site-packages/httpx/_urlparse.py", line 346, in normalize_port
      port_as_int = int(port)
  ValueError: invalid literal for int() with base 10: '7ac0::'

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/runner/.ansible/tmp/ansible-tmp-1715768974.3496563-65-93302956278022/AnsiballZ_azure_rm_virtualmachine_info.py", line 107, in <module>
      _ansiballz_main()
    File "/runner/.ansible/tmp/ansible-tmp-1715768974.3496563-65-93302956278022/AnsiballZ_azure_rm_virtualmachine_info.py", line 99, in _ansiballz_main
      invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
    File "/runner/.ansible/tmp/ansible-tmp-1715768974.3496563-65-93302956278022/AnsiballZ_azure_rm_virtualmachine_info.py", line 47, in invoke_module
      runpy.run_module(mod_name='ansible_collections.azure.azcollection.plugins.modules.azure_rm_virtualmachine_info', init_globals=dict(_module_fqn='ansible_collections.azure.azcollection.plugins.modules.azure_rm_virtualmachine_info', _modlib_path=modlib_path),
    File "/usr/lib64/python3.9/runpy.py", line 225, in run_module
      return _run_module_code(code, init_globals, run_name, mod_spec)
    File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code
      _run_code(code, mod_globals, init_globals,
    File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
      exec(code, run_globals)
    File "/tmp/ansible_azure.azcollection.azure_rm_virtualmachine_info_payload_ln1cx03d/ansible_azure.azcollection.azure_rm_virtualmachine_info_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_virtualmachine_info.py", line 335, in <module>
    File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
    File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
    File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
    File "<frozen zipimport>", line 259, in load_module
    File "/tmp/ansible_azure.azcollection.azure_rm_virtualmachine_info_payload_ln1cx03d/ansible_azure.azcollection.azure_rm_virtualmachine_info_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 290, in <module>
    File "/usr/local/lib/python3.9/site-packages/msgraph/__init__.py", line 5, in <module>
      from .graph_request_adapter import GraphRequestAdapter
    File "/usr/local/lib/python3.9/site-packages/msgraph/graph_request_adapter.py", line 5, in <module>
      from msgraph_core import APIVersion, BaseGraphRequestAdapter, GraphClientFactory
    File "/usr/local/lib/python3.9/site-packages/msgraph_core/__init__.py", line 14, in <module>
      from .base_graph_request_adapter import BaseGraphRequestAdapter
    File "/usr/local/lib/python3.9/site-packages/msgraph_core/base_graph_request_adapter.py", line 9, in <module>
      from kiota_http.httpx_request_adapter import HttpxRequestAdapter
    File "/usr/local/lib/python3.9/site-packages/kiota_http/httpx_request_adapter.py", line 58, in <module>
      class HttpxRequestAdapter(RequestAdapter, Generic[ModelType]):
    File "/usr/local/lib/python3.9/site-packages/kiota_http/httpx_request_adapter.py", line 69, in HttpxRequestAdapter
      http_client: httpx.AsyncClient = KiotaClientFactory.create_with_default_middleware(),
    File "/usr/local/lib/python3.9/site-packages/kiota_http/kiota_client_factory.py", line 61, in create_with_default_middleware
      kiota_async_client = KiotaClientFactory.get_default_client() if client is None else client
    File "/usr/local/lib/python3.9/site-packages/kiota_http/kiota_client_factory.py", line 43, in get_default_client
      return httpx.AsyncClient(timeout=timeout, http2=True)
    File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1442, in __init__
      self._mounts: dict[URLPattern, AsyncBaseTransport | None] = {
    File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1443, in <dictcomp>
      URLPattern(key): None
    File "/usr/local/lib/python3.9/site-packages/httpx/_utils.py", line 370, in __init__
      url = URL(pattern)
    File "/usr/local/lib/python3.9/site-packages/httpx/_urls.py", line 115, in __init__
      self._uri_reference = urlparse(url, **kwargs)
    File "/usr/local/lib/python3.9/site-packages/httpx/_urlparse.py", line 248, in urlparse
      parsed_port: int | None = normalize_port(port, scheme)
    File "/usr/local/lib/python3.9/site-packages/httpx/_urlparse.py", line 348, in normalize_port
      raise InvalidURL(f"Invalid port: {port!r}")
  httpx.InvalidURL: Invalid port: 'BBBB::'
msg: |-
  MODULE FAILURE
  See stdout/stderr for the exact error
rc: 1
Fred-sun commented 4 months ago

@barnabas-kurdi Judging from your error, it is caused by the mismatch between the IP address and port you specified（ httpx.InvalidURL: Invalid port: 'BBBB::'）. Could you please check it again? Thank you!

Fred-sun commented 4 months ago

@barnabas-kurdi And I think this problem should not be problem of azure - azcollection this repo, and should be the problem of ansible this platform, it should be in "https://github.com/ansible/ansible/issues"

barnabas-kurdi commented 4 months ago

@Fred-sun Thank you for your response! The thing is, I specified no address when invoking the module whatsoever, so I am not sure where and why the address is pulled from (I'm thinking the modules are using it in the API requests sent to Azure?).

The ansible.builtin.uri module (and other builtin modules) works perfectly when using the EE I built, this is why I assumed that this is not a general ansible platform problem, but an issue with the azcollection in particular. I get the same error when using any module from the collection, no matter what module arguments I try.

Fred-sun commented 4 months ago

@barnabas-kurdi I will recheck it. thank you!

barnabas-kurdi commented 3 months ago

@Fred-sun I have resolved the issue: turns out the problem was not with a dual stack address, but an ipv6 address set in the no_proxy environment variable.

no_proxy=...,AAAA:BBBB::/48,...

Once I unset the variable for the module invocation, it was working fine.

I have found this same issue in the httpx repo, however it seems to have been solved in a pull request in 2023 (https://github.com/encode/httpx/pull/2659) and the azure collection is using the newest version of httpx, so I am not sure what gives. It might be worth some time investment to figure out what might be causing the issue, since its source is not immediately apparent.

Fred-sun commented 3 months ago

@barnabas-kurdi Thank you for your feedback, we will investigate and find out the cause and fix it. Thank you!