microsoft / OMS-Agent-for-Linux

http://www.microsoft.com/oms
Other
411 stars 309 forks source link

OMSAgent keeps removing everything under /tmp folder #123

Closed jarig closed 8 years ago

jarig commented 8 years ago

For some reason omsagent keeps running following file: /etc/opt/microsoft/scx/conf/sudodir/dh_builddeb

Which includes following lines

complex_doit("find $tmp -name $_ | xargs rm -rf")

Cause of that /tmp is constantly wiped and none of other applications can normally run while omsagent is running.

jarig commented 8 years ago

Seems it's not dh_builddeb but, still find that runs rm after that.

Here is what I've managed to gather using auditd

type=PATH msg=audit(05/07/16 14:30:03.364:1099645) : item=1 name=test2 inode=2490388 dev=08:02 mode=file,644 ouid=root ogid=root rdev=00:00 type=PATH msg=audit(05/07/16 14:30:03.364:1099645) : item=0 name=/tmp inode=2490369 dev=08:02 mode=dir,sticky,777 ouid=root ogid=root rdev=00:00 type=CWD msg=audit(05/07/16 14:30:03.364:1099645) : cwd=/tmp type=SYSCALL msg=audit(05/07/16 14:30:03.364:1099645) : arch=x86_64 syscall=unlinkat success=yes exit=0 a0=ffffffffffffff9c a1=d71968 a2=0 a3=0 items=2 ppid=27719 pid=27729 auid=root uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=1584 comm=find exe=/usr/bin/find key=(null)

Parent process:

type=PATH msg=audit(05/07/16 14:30:03.360:1099644) : item=1 name=/tmp/.clean inode=2491092 dev=08:02 mode=file,644 ouid=root ogid=root rdev=00:00 type=PATH msg=audit(05/07/16 14:30:03.360:1099644) : item=0 name=/tmp/ inode=2490369 dev=08:02 mode=dir,sticky,777 ouid=root ogid=root rdev=00:00 type=CWD msg=audit(05/07/16 14:30:03.360:1099644) : cwd=/tmp type=SYSCALL msg=audit(05/07/16 14:30:03.360:1099644) : arch=x86_64 syscall=open success=yes exit=3 a0=7895f8 a1=c1 a2=1b6 a3=0 items=2 ppid=15997 pid=27719 auid=root uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=1584 comm=checkroot-bootc exe=/bin/dash key=(null)

Can't figure out who runs checkroot-bootclean, but for sure it's coming somewhere from OMS as once disabled such issue not happening.

jarig commented 8 years ago

Finally I've found whole trace of this find

init,1
  `-omiserver,4200 --configfile=/etc/opt/omi/conf/omiserver.conf -d
      `-omiagent,62752 10 13 --destdir / --providerdir /opt/omi/lib --idletimeout 90 --loglevel WARNING
          `-python,63608 /opt/microsoft/omsconfig/Scripts/client.py 6
              `-sh,64154 -c sudo /opt/microsoft/omsconfig/Scripts/OMSServiceStat.sh rcS
                  `-sudo,64155 /opt/microsoft/omsconfig/Scripts/OMSServiceStat.sh rcS
                      `-OMSServiceStat.,64156 /opt/microsoft/omsconfig/Scripts/OMSServiceStat.sh rcS
                          `-rc,64157 /etc/init.d/rc S
                              `-startpar,64163 -p 4 -t 20 -T 3 -M boot -P N -R S
                                  `-checkroot-bootc,11916 /etc/init.d/checkroot-bootclean.sh start
                                      `-find,11926 . -depth -xdev ! -name . ! ( -path ./lost+found -uid 0 ) ! ( -path ./quota.user -uid 0 ) ! ( -path ./aquota.user -uid 0 ) ! ( -path ./quota.group -uid 0 ) ! ( -path ./aquota.group -uid 0 ) ! ( -path ./.journal -uid 0 ) ! ( -path ./.clean -uid 0 ) ! ( -path ./...security* -uid 0 ) ! -type d -delete
jarig commented 8 years ago

I've found that issue originates from nxService.py from https://github.com/Microsoft/PowerShell-DSC-for-Linux: https://github.com/Microsoft/PowerShell-DSC-for-Linux/blob/4d3a586d3ae658ba59fc1dd2bf88f276cd1e9ee3/Providers/Scripts/2.6x-2.7x/Scripts/nxService.py#L1605

It has wrong grep command that outputs list of services to get status for, and one of them is rcS. If call service rcS status it causes system to re-initialize boot procedures and cleanup /tmp folder. More details in the pull request I've made to that repository.

I suppose this one also have to be opened to pick up changes once they are merged/fixed in PowerShell-DSC-for-Linux.

jeffaco commented 8 years ago

Very cool that you fixed this and it was committed. Thanks!

I'm going to go ahead and close this issue since it's resolved now. Thanks very much for your assistance with this. If this issue is still outstanding or you have questions about it, feel free to reopen this issue.