search

ansible-collections / community.general

Ansible Community General Collection
https://galaxy.ansible.com/ui/repo/published/community/general/
GNU General Public License v3.0
822 stars 1.52k forks source link

cache redis :: Error -2 connecting to localhost:6379. Name or service not known #7814

Closed adriansev closed 9 months ago

adriansev commented 9 months ago

Summary

After updating to:

pip3 freeze | grep ansible
ansible==9.1.0
ansible-compat==4.1.10
ansible-core==2.16.2
ansible-inventory==0.6.4
ansible-lint==6.22.1

the redis connection for fact caching is broken:

ansible-inventory --list 
ERROR! Unexpected Exception, this is probably a bug: Error -2 connecting to "localhost:6379. Name or service not known.
to see the full traceback, use -vvv

this is my simple declaration or redis connection that used to work https://github.com/adriansev/ansible_iss/blob/master/ansible.cfg#L49

Any idea about this? Thanks a lot!

Issue Type

Bug Report

Component Name

redis

Ansible Version

ansible --version
ansible [core 2.16.2]
  config file = /home/adrian/ansible/ansible.cfg
  configured module search path = ['/home/adrian/ansible/my_modules']
  ansible python module location = /home/adrian/.local/lib/python3.11/site-packages/ansible
  ansible collection location = /home/adrian/ansible:/home/adrian/.local/lib64/python3.11/site-packages
  executable location = /home/adrian/.local/bin/ansible
  python version = 3.11.6 (main, Oct  3 2023, 00:00:00) [GCC 13.2.1 20230728 (Red Hat 13.2.1-1)] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

Community.general Version

ansible-galaxy collection list community.general

# /home/adrian/.local/lib64/python3.11/site-packages/ansible_collections
Collection        Version
----------------- -------
community.general 8.1.0  

# /home/adrian/ansible/ansible_collections
Collection        Version
----------------- -------
community.general 8.2.0

Configuration

ansible-config dump --only-changed
ANSIBLE_HOME(env: ANSIBLE_HOME) = /home/adrian/ansible
ANSIBLE_NOCOWS(/home/adrian/ansible/ansible.cfg) = True
BECOME_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/become" }}']
CACHE_PLUGIN(/home/adrian/ansible/ansible.cfg) = community.general.redis
CACHE_PLUGIN_CONNECTION(/home/adrian/ansible/ansible.cfg) = "localhost:6379:0:"
CACHE_PLUGIN_PREFIX(/home/adrian/ansible/ansible.cfg) = ""
CACHE_PLUGIN_TIMEOUT(/home/adrian/ansible/ansible.cfg) = 86400
CALLBACKS_ENABLED(/home/adrian/ansible/ansible.cfg) = ['debug', 'log_plays', 'timer', 'profile_tasks', 'profile_roles']
COLLECTIONS_PATHS(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible', '/home/adrian/.local/lib64/python3.11/site-packages']
COLLECTIONS_SCAN_SYS_PATH(/home/adrian/ansible/ansible.cfg) = False
CONFIG_FILE() = /home/adrian/ansible/ansible.cfg
DEFAULT_ACTION_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/action" }}']
DEFAULT_CACHE_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/cache" }}']
DEFAULT_CALLBACK_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/callback" }}']
DEFAULT_CLICONF_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/cliconf" }}']
DEFAULT_CONNECTION_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/connection" }}']
DEFAULT_DEBUG(/home/adrian/ansible/ansible.cfg) = False
DEFAULT_EXECUTABLE(/home/adrian/ansible/ansible.cfg) = /bin/bash
DEFAULT_FACT_PATH(/home/adrian/ansible/ansible.cfg) = ~/ansible/facts.d
DEFAULT_FILTER_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/filter" }}']
DEFAULT_FORKS(/home/adrian/ansible/ansible.cfg) = 50
DEFAULT_GATHERING(/home/adrian/ansible/ansible.cfg) = smart
DEFAULT_GATHER_SUBSET(/home/adrian/ansible/ansible.cfg) = ['all', '!facter', '!ohai']
DEFAULT_GATHER_TIMEOUT(/home/adrian/ansible/ansible.cfg) = 30
ANSIBLE_HOME(env: ANSIBLE_HOME) = /home/adrian/ansible
ANSIBLE_NOCOWS(/home/adrian/ansible/ansible.cfg) = True
BECOME_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/become" }}']
CACHE_PLUGIN(/home/adrian/ansible/ansible.cfg) = community.general.redis
CACHE_PLUGIN_CONNECTION(/home/adrian/ansible/ansible.cfg) = "localhost:6379:0:"
CACHE_PLUGIN_PREFIX(/home/adrian/ansible/ansible.cfg) = ""
CACHE_PLUGIN_TIMEOUT(/home/adrian/ansible/ansible.cfg) = 86400
CALLBACKS_ENABLED(/home/adrian/ansible/ansible.cfg) = ['debug', 'log_plays', 'timer', 'profile_tasks', 'profile_roles']
COLLECTIONS_PATHS(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible', '/home/adrian/.local/lib64/python3.11/site-packages']
COLLECTIONS_SCAN_SYS_PATH(/home/adrian/ansible/ansible.cfg) = False
CONFIG_FILE() = /home/adrian/ansible/ansible.cfg
DEFAULT_ACTION_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/action" }}']
DEFAULT_CACHE_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/cache" }}']
DEFAULT_CALLBACK_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/callback" }}']
DEFAULT_CLICONF_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/cliconf" }}']
DEFAULT_CONNECTION_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/connection" }}']
DEFAULT_DEBUG(/home/adrian/ansible/ansible.cfg) = False
DEFAULT_EXECUTABLE(/home/adrian/ansible/ansible.cfg) = /bin/bash
DEFAULT_FACT_PATH(/home/adrian/ansible/ansible.cfg) = ~/ansible/facts.d
DEFAULT_FILTER_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/filter" }}']
DEFAULT_FORKS(/home/adrian/ansible/ansible.cfg) = 50
DEFAULT_GATHERING(/home/adrian/ansible/ansible.cfg) = smart
DEFAULT_GATHER_SUBSET(/home/adrian/ansible/ansible.cfg) = ['all', '!facter', '!ohai']
DEFAULT_GATHER_TIMEOUT(/home/adrian/ansible/ansible.cfg) = 30
DEFAULT_HASH_BEHAVIOUR(/home/adrian/ansible/ansible.cfg) = replace
DEFAULT_HOST_LIST(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/inventory']
DEFAULT_HTTPAPI_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/httpapi" }}']
DEFAULT_INVENTORY_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/inventory" }}']
DEFAULT_LOAD_CALLBACK_PLUGINS(/home/adrian/ansible/ansible.cfg) = True
DEFAULT_LOCAL_TMP(/home/adrian/ansible/ansible.cfg) = /home/adrian/ansible/tmp/ansible-local-2582937wx80psko
DEFAULT_LOOKUP_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/lookup" }}']
DEFAULT_MANAGED_STR(/home/adrian/ansible/ansible.cfg) = !!!ANSIBLE MANAGED {file}/{host}!!!
DEFAULT_MODULE_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/my_modules']
DEFAULT_MODULE_UTILS_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/my_module_utils']
DEFAULT_NETCONF_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/netconf" }}']
DEFAULT_NO_LOG(/home/adrian/ansible/ansible.cfg) = False
DEFAULT_NO_TARGET_SYSLOG(/home/adrian/ansible/ansible.cfg) = False
DEFAULT_REMOTE_PORT(/home/adrian/ansible/ansible.cfg) = 60000
DEFAULT_REMOTE_USER(/home/adrian/ansible/ansible.cfg) = root
DEFAULT_ROLES_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/roles']
DEFAULT_STDOUT_CALLBACK(/home/adrian/ansible/ansible.cfg) = debug
DEFAULT_STRATEGY(/home/adrian/ansible/ansible.cfg) = linear
DEFAULT_STRATEGY_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/strategy" }}']
DEFAULT_TERMINAL_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/terminal" }}']
DEFAULT_TEST_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/test" }}']
DEFAULT_TIMEOUT(/home/adrian/ansible/ansible.cfg) = 10
DEFAULT_VARS_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/vars" }}']
DEPRECATION_WARNINGS(/home/adrian/ansible/ansible.cfg) = False
DISPLAY_ARGS_TO_STDOUT(/home/adrian/ansible/ansible.cfg) = False
DISPLAY_SKIPPED_HOSTS(/home/adrian/ansible/ansible.cfg) = False
DOC_FRAGMENT_PLUGIN_PATH(/home/adrian/ansible/ansible.cfg) = ['/home/adrian/ansible/{{ ANSIBLE_HOME ~ "/plugins/doc_fragments" }}']
EDITOR(env: EDITOR) = mcedit
GALAXY_CACHE_DIR(/home/adrian/ansible/ansible.cfg) = /home/adrian/ansible/galaxy_cache
GALAXY_TOKEN_PATH(/home/adrian/ansible/ansible.cfg) = /home/adrian/ansible/galaxy_token
HOST_KEY_CHECKING(/home/adrian/ansible/ansible.cfg) = False
INTERPRETER_PYTHON(/home/adrian/ansible/ansible.cfg) = auto_silent
INVENTORY_CACHE_ENABLED(/home/adrian/ansible/ansible.cfg) = True
INVENTORY_CACHE_PLUGIN(/home/adrian/ansible/ansible.cfg) = ansible.builtin.memory
INVENTORY_ENABLED(/home/adrian/ansible/ansible.cfg) = ['ini', 'yaml', 'host_list', 'script', 'auto']
PARAMIKO_HOST_KEY_AUTO_ADD(/home/adrian/ansible/ansible.cfg) = True
PERSISTENT_COMMAND_TIMEOUT(/home/adrian/ansible/ansible.cfg) = 10
PERSISTENT_CONNECT_RETRY_TIMEOUT(/home/adrian/ansible/ansible.cfg) = 15
PERSISTENT_CONNECT_TIMEOUT(/home/adrian/ansible/ansible.cfg) = 30
SYSTEM_WARNINGS(/home/adrian/ansible/ansible.cfg) = True

OS / Environment

Fedora 39

Steps to Reproduce

just the simplest of ansible-inventory --list see the complete post above

Expected Results

I expected no errors (until yesterday everything worked then i updated all ansible related components)

Actual Results

ansible-inventory --list -vvvvvv
ansible-inventory [core 2.16.2]
  config file = /home/adrian/ansible/ansible.cfg
  configured module search path = ['/home/adrian/ansible/my_modules']
  ansible python module location = /home/adrian/.local/lib/python3.11/site-packages/ansible
  ansible collection location = /home/adrian/ansible:/home/adrian/.local/lib64/python3.11/site-packages
  executable location = /home/adrian/.local/bin/ansible-inventory
  python version = 3.11.6 (main, Oct  3 2023, 00:00:00) [GCC 13.2.1 20230728 (Red Hat 13.2.1-1)] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True
Using /home/adrian/ansible/ansible.cfg as config file
setting up inventory plugins
Loading collection ansible.builtin from 
Parsed /home/adrian/ansible/inventory/eos_fst inventory source with ini plugin
setting up inventory plugins
Parsed /home/adrian/ansible/inventory/hosts inventory source with ini plugin
setting up inventory plugins
Parsed /home/adrian/ansible/inventory/wn_alien inventory source with ini plugin
setting up inventory plugins
Parsed /home/adrian/ansible/inventory/wn_issaf inventory source with ini plugin
setting up inventory plugins
Parsed /home/adrian/ansible/inventory/wn_umd inventory source with ini plugin
Loading collection community.general from /home/adrian/ansible/ansible_collections/community/general
Redis connection: Redis<ConnectionPool<Connection<host="localhost,port=6379,db=0>>>
ERROR! Unexpected Exception, this is probably a bug: Error -2 connecting to "localhost:6379. Name or service not known.
the full traceback was:

Traceback (most recent call last):
  File "/home/adrian/.local/lib/python3.11/site-packages/redis/connection.py", line 264, in connect
    sock = self.retry.call_with_retry(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adrian/.local/lib/python3.11/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
           ^^^^
  File "/home/adrian/.local/lib/python3.11/site-packages/redis/connection.py", line 265, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
            ^^^^^^^^^^^^^^^
  File "/home/adrian/.local/lib/python3.11/site-packages/redis/connection.py", line 595, in _connect
    for res in socket.getaddrinfo(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/socket.py", line 962, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/adrian/.local/lib/python3.11/site-packages/ansible/cli/__init__.py", line 659, in cli_executor
    exit_code = cli.run()
                ^^^^^^^^^
  File "/home/adrian/.local/lib/python3.11/site-packages/ansible/cli/inventory.py", line 161, in run
    results = self.json_inventory(top)
              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adrian/.local/lib/python3.11/site-packages/ansible/cli/inventory.py", line 333, in json_inventory
    hvars = self._get_host_variables(host)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adrian/.local/lib/python3.11/site-packages/ansible/cli/inventory.py", line 239, in _get_host_variables
    hostvars = self.vm.get_vars(host=host, include_hostvars=False, stage='all')
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adrian/.local/lib/python3.11/site-packages/ansible/vars/manager.py", line 313, in get_vars
    facts = wrap_var(self._fact_cache.get(host.name, {}))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen _collections_abc>", line 774, in get
  File "/home/adrian/.local/lib/python3.11/site-packages/ansible/vars/fact_cache.py", line 31, in __getitem__
    if not self._plugin.contains(key):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adrian/ansible/ansible_collections/community/general/plugins/cache/redis.py", line 210, in contains
    self._expire_keys()
  File "/home/adrian/ansible/ansible_collections/community/general/plugins/cache/redis.py", line 203, in _expire_keys
    self._db.zremrangebyscore(self._keys_set, 0, expiry_age)
  File "/home/adrian/.local/lib/python3.11/site-packages/redis/commands/core.py", line 4772, in zremrangebyscore
    return self.execute_command("ZREMRANGEBYSCORE", name, min, max)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adrian/.local/lib/python3.11/site-packages/redis/client.py", line 533, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adrian/.local/lib/python3.11/site-packages/redis/connection.py", line 1086, in get_connection
    connection.connect()
  File "/home/adrian/.local/lib/python3.11/site-packages/redis/connection.py", line 270, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error -2 connecting to "localhost:6379. Name or service not known.

Code of Conduct

ansibullbot commented 9 months ago

Files identified in the description:

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot commented 9 months ago

cc @slok click here for bot help

felixfontein commented 9 months ago

!component =plugins/cache/redis.py

ansibullbot commented 9 months ago

Files identified in the description:

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

felixfontein commented 9 months ago

My guess is that this is related to the redis Python library. Which version did you use before (when it worked), and which version do you use now?

adriansev commented 9 months ago

Well, i have no idea but i just tried many redis versions (down to 4.3.0 from 8 may 2022) and i get the same error .. I think is something really weird with the message:

ERROR! Unexpected Exception, this is probably a bug: Error -2 connecting to "localhost:6379. Name or service not known.

notice the unpaired quotes so i removed the quotes from my connection setting:

 fact_caching = community.general.redis
-fact_caching_connection = "localhost:6379:0:"
+fact_caching_connection = localhost:6379:0:

and the problem is solved .. so, at this moment i'm inclined to think that there is a problem with the community.general.redis parsing of fact_caching_connection value .. OTOH looking in https://github.com/ansible-collections/community.general/blame/main/plugins/cache/redis.py in the blame list i see no relevant recent modification .. i only noted this line: https://github.com/ansible-collections/community.general/blob/main/plugins/cache/redis.py#L150 where it is noted the format for _uri that is quoted .. So, i have no idea about the actual bug ... Now the question is: the fact the the connection declaration don't like anymore the quotes is a bug? Thanks a lot!

felixfontein commented 9 months ago

Good spot with the unpaired quote! The parsing of ansible.cfg is done by ansible-core, so maybe you changed the version of that one, and parsing behavior of that one changed? (Because as you noticed, the plugin didn't changed.)

adriansev commented 9 months ago

one more thing about parsing: i had fact_caching_prefix = "" and after the above solution i just found out that i have the host keys in redis with the form:

""issaf-0-4
""issaf-0-5

notice the "" in front... so for certain the parsing and interpretation of value definitions in ansible.cfg was changed

adriansev commented 9 months ago

so, for reference, @bcoca mentioned this fix for this known problem. Given that it is known problem and it is fixed i will close this issue.