[BUG]I'm passing the script through salt base64 to the target machine and running it, but salt-master EventPublisher CPU 100%,Whether this thread supports extension

[764276020]

Description I'm passing the script through salt base64 to the target machine and running it, but salt-master EventPublisher CPU 100%,Whether this thread supports extension?

We use saltstack as the mission system, salt-minion number 8000+

Setup (Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)

Please be as specific as possible and give set-up details.

• [ ] on-prem machine
• [ √] VM (Virtualbox, KVM, etc. please specify)
• [ ] VM running on a cloud service, please be explicit and add details
• [ ] container (Kubernetes, Docker, containerd, etc. please specify)
• [ ] or a combination, please be explicit
• [ ] jails if it is FreeBSD
• [ ] classic packaging
• [ ] onedir packaging
• [ ] used bootstrap to install

Steps to Reproduce the behavior (Include debug logs if possible and relevant)

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Versions Report

salt --versions-report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) ```yaml PASTE HERE ```

Additional context Add any other context about the problem here.

[welcome[bot]]

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

• Community Wiki
• Salt’s Contributor Guide
• Join our Community Slack
• IRC on LiberaChat
• Salt Project YouTube channel
• Salt Project Twitch channel

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at saltproject@vmware.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

[764276020]

[root@saltmaster-m192-49 tmp]# salt --versions-report Salt Version: Salt: 3005.1

Dependency Versions: cffi: Not Installed cherrypy: unknown dateutil: Not Installed docker-py: Not Installed gitdb: Not Installed gitpython: Not Installed Jinja2: 2.8.1 libgit2: Not Installed M2Crypto: 0.33.0 Mako: Not Installed msgpack: 0.6.2 msgpack-pure: Not Installed mysql-python: Not Installed pycparser: Not Installed pycrypto: Not Installed pycryptodome: Not Installed pygit2: Not Installed Python: 3.6.8 (default, Nov 16 2020, 16:55:22) python-gnupg: Not Installed PyYAML: 3.11 PyZMQ: 18.0.1 smmap: Not Installed timelib: Not Installed Tornado: 4.5.3 ZMQ: 4.1.4

System Versions: dist: centos 7 Core locale: UTF-8 machine: x86_64 release: 3.10.0-1127.19.1.el7.x86_64 system: Linux version: CentOS Linux 7 Core

[764276020]

@garethgreenaway Could you answer that, please!

[OrangeDog]

8000 is a lot of minions. It's very easy to overload a single master if you're not careful.

I don't understand what "passing the script through salt base64" means. Please provide actual reproduction of the issue, including relevant config, sls files, and the command(s) you are running.

[764276020]

8000 is a lot of minions. It's very easy to overload a single master if you're not careful.

I don't understand what "passing the script through salt base64" means. Please provide actual reproduction of the issue, including relevant config, sls files, and the command(s) you are running.

I suspect eventpublisher cpu 100% is shell script in hashutil.base64_decodefile

I used two salt-syndic nodes， But only the top master has problems

[764276020]

8000 is a lot of minions. It's very easy to overload a single master if you're not careful. I don't understand what "passing the script through salt base64" means. Please provide actual reproduction of the issue, including relevant config, sls files, and the command(s) you are running.

I suspect eventpublisher cpu 100% is shell script in hashutil.base64_decodefile

I used two salt-syndic nodes， But only the top master has problems

like this

Currently don't know how to expand

[OrangeDog]

Please provide actual reproduction of the issue, including relevant config, sls files, and the command(s) you are running.

[764276020]

请提供问题的实际重现，包括相关配置、sls 文件和您正在运行的命令。

i am executing salt '*' test.ping

[OrangeDog]

How is that "passing the script through salt base64 to the target machine and running it"?

What are you actually doing? What script? What config? How are you passing it and running it?

[764276020]

How is that "passing the script through salt base64 to the target machine and running it"?

What are you actually doing? What script? What config? How are you passing it and running it?

I'm sorry，i mean saltstack minion too many, salt top master event pubulisher single process(thread) performance is not enough, I want to extend it, or is there any other way to solve it

[764276020]

How is that "passing the script through salt base64 to the target machine and running it"?

What are you actually doing? What script? What config? How are you passing it and running it?

even running salt '*' test.ping causes salt top master eventpublisher thread cpu 100%.
I want to ask is there a solution?

[OrangeDog]

Possible duplicate: #61845

[764276020]

Possible duplicate: #61845

no, i see salt top master eventpublisher cpu 100%, I want to extend it, or is there any other way to solve it

[764276020]

I've tried extending salt-top-master-eventpublisher-processing to multiple processes, but it doesn't seem to work

[764276020]

Possible duplicate: #61845

no, i see salt top master eventpublisher cpu 100%, I want to extend it, or is there any other way to solve it

may be salt minion too many lead to insufficient performance，Is salt top master eventpublisher possible to make it run in multiple processes

[764276020]

Excuse me, is there any progress?

[dwoz]

Is zmq_filtering enabled?

[764276020]

Is zmq_filtering enabled?

not enable

configuration: interface: 192.168.192.49 worker_threads: 8 auto_accept: False keysize: 2048 master_id: saltmaster-m192-49 keep_jobs: 24 job_cache: True cachedir: /home/data/salt/cache/master sqlite_queue_dir: /home/data/salt/cache/master/queues sock_pool_size: 10240 tcp_master_workers: 10240 ipc_mode: ipc default_include: master.d/*.yml file_roots: base:

• /home/data/salt/data/states
• /home/data/salt/data/services pillar_roots: base:
• /home/data/salt/data/pillar

order_masters: True

rest_cherrypy: port: 8888 disable_ssl: 1 host: 192.168.192.49 thread_pool: 2048 socket_queue_size: 1024 debug: True log_error_file: /home/data/salt/var/log/api/error.log log_access_file: /home/data/salt/var/log/api/access.log expire_responses: False

mysql_auth: ....

gather_job_timeout: 120 timeout: 60 show_timeout: True show_jid: True

log_level: warn