search

openSUSE / salt

openSUSE and SUSE patches and backports for SaltStack
Apache License 2.0
22 stars 52 forks source link

Revert the change making reactor less blocking (bsc#1230322) #680

Closed vzhestkov closed 3 weeks ago

vzhestkov commented 1 month ago

What does this PR do?

Revert of https://github.com/openSUSE/salt/pull/634

This reverts commit 0d35f09288700f5c961567442c3fcc25838b8de4.

With the PR mentioned it causes some conflict with gitfs integration which could start failing under high workload.

As the speed improvement of the original PR is not very high it makes sense just to revert this change to prevent such fails.

What issues does this PR fix or reference?

Tracks: https://github.com/SUSE/spacewalk/issues/25225

Previous Behavior

Possible tracebacks on processing reactor like the following:

2024-09-02 09:28:06,051 [salt.utils.reactor:79  ][ERROR   ][3743] Failed to render "/srv/reactor/auto_update/reboot_system.sls": 
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 493, in __init__
    self.new = self.init_remote()
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 1897, in init_remote
    self.repo = pygit2.Repository(self._cachedir)
  File "/usr/lib64/python3.6/site-packages/pygit2/repository.py", line 1245, in __init__
    path_backend = init_file_backend(path)
_pygit2.GitError: /var/cache/salt/master/gitfs/spZu6W+WqNHwM9bcSgOjZZO4ndNjb5p0S+7BXuLLJ9M=/_: failed to stat '/root/.gitconfig'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/templates.py", line 477, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3.6/site-packages/jinja2/asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.6/site-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python3.6/site-packages/salt/utils/jinja.py", line 162, in get_source
    self.check_cache(_template)
  File "/usr/lib/python3.6/site-packages/salt/utils/jinja.py", line 115, in check_cache
    ret = self.cache_file(template)
  File "/usr/lib/python3.6/site-packages/salt/utils/jinja.py", line 107, in cache_file
    fcl = self.file_client()
  File "/usr/lib/python3.6/site-packages/salt/utils/jinja.py", line 97, in file_client
    self.opts, self.pillar_rend
  File "/usr/lib/python3.6/site-packages/salt/fileclient.py", line 56, in get_file_client
    )(opts)
  File "/usr/lib/python3.6/site-packages/salt/fileclient.py", line 1462, in __init__
    self.channel = salt.fileserver.FSChan(opts)
  File "/usr/lib/python3.6/site-packages/salt/fileserver/__init__.py", line 865, in __init__
    self.fs.init()
  File "/usr/lib/python3.6/site-packages/salt/fileserver/__init__.py", line 540, in init
    self.servers[fstr]()
  File "/usr/lib/python3.6/site-packages/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/loader/lazy.py", line 1234, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/loader/lazy.py", line 1249, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/fileserver/gitfs.py", line 169, in init
    _gitfs()
  File "/usr/lib/python3.6/site-packages/salt/fileserver/gitfs.py", line 89, in _gitfs
    init_remotes=init_remotes,
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 3019, in __new__
    init_remotes=init_remotes,
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 2429, in __init__
    global_only,
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 2492, in init_remotes
    self.role,
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 1627, in __init__
    role,
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 501, in __init__
    failhard(self.role)
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 220, in failhard
    raise FileserverConfigError("Failed to load {}".format(role))
salt.exceptions.FileserverConfigError: Failed to load gitfs

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/reactor.py", line 70, in render_reaction
    res = self.render_template(fn_, tag=tag, data=data)
  File "/usr/lib/python3.6/site-packages/salt/state.py", line 393, in render_template
    **kwargs,
  File "/usr/lib/python3.6/site-packages/salt/template.py", line 99, in compile_template
    ret = render(input_data, saltenv, sls, **render_kwargs)
  File "/usr/lib/python3.6/site-packages/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/loader/lazy.py", line 1234, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/loader/lazy.py", line 1249, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/renderers/jinja.py", line 75, in render
    **kws
  File "/usr/lib/python3.6/site-packages/salt/utils/templates.py", line 219, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/usr/lib/python3.6/site-packages/salt/utils/templates.py", line 524, in render_jinja_tmpl
    "Jinja error: {}{}".format(exc, out), line, tmplstr, trace=tracestr
salt.exceptions.SaltRenderError: Jinja error: Failed to load gitfs
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 493, in __init__
    self.new = self.init_remote()
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 1897, in init_remote
    self.repo = pygit2.Repository(self._cachedir)
  File "/usr/lib64/python3.6/site-packages/pygit2/repository.py", line 1245, in __init__
    path_backend = init_file_backend(path)
_pygit2.GitError: /var/cache/salt/master/gitfs/spZu6W+WqNHwM9bcSgOjZZO4ndNjb5p0S+7BXuLLJ9M=/_: failed to stat '/root/.gitconfig'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/templates.py", line 477, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3.6/site-packages/jinja2/asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.6/site-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python3.6/site-packages/salt/utils/jinja.py", line 162, in get_source
    self.check_cache(_template)
  File "/usr/lib/python3.6/site-packages/salt/utils/jinja.py", line 115, in check_cache
    ret = self.cache_file(template)
  File "/usr/lib/python3.6/site-packages/salt/utils/jinja.py", line 107, in cache_file
    fcl = self.file_client()
  File "/usr/lib/python3.6/site-packages/salt/utils/jinja.py", line 97, in file_client
    self.opts, self.pillar_rend
  File "/usr/lib/python3.6/site-packages/salt/fileclient.py", line 56, in get_file_client
    )(opts)
  File "/usr/lib/python3.6/site-packages/salt/fileclient.py", line 1462, in __init__
    self.channel = salt.fileserver.FSChan(opts)
  File "/usr/lib/python3.6/site-packages/salt/fileserver/__init__.py", line 865, in __init__
    self.fs.init()
  File "/usr/lib/python3.6/site-packages/salt/fileserver/__init__.py", line 540, in init
    self.servers[fstr]()
  File "/usr/lib/python3.6/site-packages/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/loader/lazy.py", line 1234, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/loader/lazy.py", line 1249, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/fileserver/gitfs.py", line 169, in init
    _gitfs()
  File "/usr/lib/python3.6/site-packages/salt/fileserver/gitfs.py", line 89, in _gitfs
    init_remotes=init_remotes,
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 3019, in __new__
    init_remotes=init_remotes,
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 2429, in __init__
    global_only,
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 2492, in init_remotes
    self.role,
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 1627, in __init__
    role,
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 501, in __init__
    failhard(self.role)
  File "/usr/lib/python3.6/site-packages/salt/utils/gitfs.py", line 220, in failhard
    raise FileserverConfigError("Failed to load {}".format(role))
salt.exceptions.FileserverConfigError: Failed to load gitfs

; line 1

New Behavior

No conflicts causing such exception

Merge requirements satisfied?

[NOTICE] Bug fixes or features added to Salt require tests.

Commits signed with GPG?

Yes/No

Please review Salt's Contributing Guide for best practices.

See GitHub's page on GPG signing for more information about signing commits with GPG.