maldet.service cannot be started

Ricky-Tigg commented 2 years ago

maldetect: v. 1.6.4; OS: Fedora x86_64; Selinux: enabled

maldet.service can be enabled but not started

Prerequisites:

$ curl -C - -LO https://www.rfxn.com/downloads/maldetect-current.tar.gz && tar xvzf maldetect-current.tar.gz; cd maldetect-1.6.4; sudo ./install.sh
Failed to enable unit: Unit file maldet.service does not exist.
(...)
# restorecon -v /usr/lib/systemd/system/maldet.service
Relabeled /usr/lib/systemd/system/maldet.service from unconfined_u:object_r:user_home_t:s0 to unconfined_u:object_r:systemd_unit_file_t:s0

To reproduce:

# systemctl --now enable maldet.service
Created symlink /etc/systemd/system/multi-user.target.wants/maldet.service → /usr/lib/systemd/system/maldet.service.
Job for maldet.service failed because the service did not take the steps required by its unit configuration.
See "systemctl status maldet.service" and "journalctl -xeu maldet.service" for details.

Logs:

$ systemctl status maldet.service
x maldet.service - Linux Malware Detect monitoring - maldet
     Loaded: loaded (/usr/lib/systemd/system/maldet.service; enabled; vendor preset: disabled)
     Active: failed (Result: protocol) since Sat 2021-12-04 14:30:43 EET; 23min ago
    Process: 28329 ExecStart=/usr/local/maldetect/maldet --monitor $default_monitor_mode (code=exited, status=0/SUCCESS)
        CPU: 212ms

Dec 04 14:30:42 fedora systemd[1]: Starting Linux Malware Detect monitoring - maldet...
Dec 04 14:30:42 fedora maldet[28329]: Linux Malware Detect v1.6.4
Dec 04 14:30:42 fedora maldet[28329]:             (C) 2002-2019, R-fx Networks <proj@rfxn.com>
Dec 04 14:30:42 fedora maldet[28329]:             (C) 2019, Ryan MacDonald <ryan@rfxn.com>
Dec 04 14:30:42 fedora maldet[28329]: This program may be freely redistributed under the terms of the GNU GPL v2
Dec 04 14:30:43 fedora maldet[28329]: maldet(28329): {mon} could not find inotifywait command, install yum package inotify-tools or download from https://github.com/rvoicilas/inotify-tools/wiki/
Dec 04 14:30:43 fedora systemd[1]: maldet.service: Can't open PID file /usr/local/maldetect/tmp/inotifywait.pid (yet?) after start: Operation not permitted
Dec 04 14:30:43 fedora systemd[1]: maldet.service: Failed with result 'protocol'.
Dec 04 14:30:43 fedora systemd[1]: Failed to start Linux Malware Detect monitoring - maldet.

journalctl_maldet.txt

mkorthof commented 2 years ago

You could try to see if this patch fixes the issue: https://github.com/rfxn/linux-malware-detect/pull/398#issuecomment-1000845157

oberling commented 2 years ago

I replaced the functions-file in 1.6.4 with this version and hoped for the best - this fixed it for my debian bullseye.

There also exists a serverfault "bugfix" for this: https://serverfault.com/questions/1077310/update-from-debian-10-to-debian-11-gone-wrong

mkorthof commented 2 years ago

I replaced the functions-file in 1.6.4 with this version and hoped for the best - this fixed it for my debian bullseye. There also exists a serverfault "bugfix" for this: https://serverfault.com/questions/1077310/update-from-debian-10-to-debian-11-gone-wrong

Any changes will be overwritten on maldet update.

To keep reapplying the patch:

Create '/etc/cron.d/maldet_inotify_patch' with contents: 0 */1 * * * root /root/src/maldet-inotify.patch.sh >/dev/null 2>&1

Create '/root/src/maldet-inotify.patch.sh' with contents:

#!/bin/sh
( cd /usr/local/maldetect && patch -N -p1 -r /dev/null < /root/src/maldet-inotify.patch ) && \
  systemctl restart maldet

Where '/root/src/maldet-inotify.patch' contains https://github.com/rfxn/linux-malware-detect/pull/398#issuecomment-1000845157

bitwanderer commented 1 year ago

I'm getting an error when applying the patch:

patch -N -p1 -r /dev/null < maldetect-patch patching file files/internals/functions Hunk #1 FAILED at 1656. 1 out of 1 hunk FAILED -- saving rejects to file /dev/null

mkorthof commented 1 year ago

I'm getting an error when applying the patch:

patch -N -p1 -r /dev/null < maldetect-patch patching file files/internals/functions Hunk #1 FAILED at 1656. 1 out of 1 hunk FAILED -- saving rejects to file /dev/null

In the meantime maldet has been updated so the patch is not needed anymore (https://github.com/rfxn/linux-malware-detect/commit/30c0fad93af59c72fc07a357ffb5b9a1162a8bab)

bitwanderer commented 1 year ago

Oh I see it's fixed in 1.6.5. Where do I get that? It's not under releases.

maldet(639406): {update} checking for available updates... maldet(639406): {update} hashing install files and checking against server... maldet(639406): {update} latest version already installed.

bitwanderer commented 1 year ago

So I tried installing 1.6.5 from the master, but after running maldet -d it goes back to 1.6.4:

maldet(656540): {update} checking for available updates... maldet(656540): {update} hashing install files and checking against server... maldet(656540): {update} version check shows latest but hash check failed, forcing update... maldet(656540): {update} verified md5sum of maldetect-current.tar.gz maldet(656540): {update} completed update v1.6.5 960304 => v1.6.4 8d5e2b, running signature updates... maldet(657154): {sigup} performing signature update check... maldet(657154): {sigup} local signature set is version 202302251165796 maldet(657154): {sigup} latest signature set already installed maldet(656540): {update} update and config import completed

Ricky-Tigg commented 1 year ago

bitwanderer | "fixed in 1.6.5. (...) It's not under releases."

Hello. In order to make sense that has to mean that a maintainner had nothing new to release. Unusual practise among developpers to indicate to users that yet nothing is ready.

bitwanderer commented 1 year ago

Web capture_28-2-2023_9817_github com

nicutor commented 1 year ago

Hi there. Today the maldet was upgraded with maldet -d and after has stopped to work:

# systemctl status maldet
● maldet.service - Linux Malware Detect monitoring - maldet
   Loaded: loaded (/usr/lib/systemd/system/maldet.service; enabled; vendor preset: disabled)
   Active: failed (Result: protocol) since Thu 2023-03-30 19:25:11 EEST; 11s ago
  Process: 510264 ExecStart=/usr/local/maldetect/maldet --monitor $default_monitor_mode (code=exited, status=0/SUCCESS)
 Main PID: 3564 (code=killed, signal=KILL)

Mar 30 19:25:10 srv3.powerhosting.com systemd[1]: Starting Linux Malware Detect monitoring - maldet...
Mar 30 19:25:10 srv3.powerhosting.com maldet[510264]: Linux Malware Detect v1.6.5
Mar 30 19:25:10 srv3.powerhosting.com maldet[510264]:             (C) 2002-2023, R-fx Networks <proj@rfxn.com>
Mar 30 19:25:10 srv3.powerhosting.com maldet[510264]:             (C) 2023, Ryan MacDonald <ryan@rfxn.com>
Mar 30 19:25:10 srv3.powerhosting.com maldet[510264]: This program may be freely redistributed under the terms of the GNU GPL v2
Mar 30 19:25:11 srv3.powerhosting.com systemd[1]: maldet.service: Can't open PID file /usr/local/maldetect/tmp/inotifywait.pid (yet?) after start: No such file or directory
Mar 30 19:25:11 srv3.powerhosting.com systemd[1]: maldet.service: Failed with result 'protocol'.
Mar 30 19:25:11 srv3.powerhosting.com systemd[1]: Failed to start Linux Malware Detect monitoring - maldet.

Linux Malware Detect v1.6.5 Rocky Linux release 8.7 (Green Obsidian)

How can I fix it?

nicutor commented 1 year ago

The same on:

CentOS Linux release 7.9.2009 (Core) Ubuntu 18.04.4 LTS (Bionic Beaver)

rfxn commented 1 year ago

@nicutor reviewing now

rfxn commented 1 year ago

@nicutor can you please provide the contents of: cat /etc/sysconfig/maldet and egrep -r ^default_monitor_mode /usr/local/maldetect*/conf.maldet

nicutor commented 1 year ago

@rfxn here is it:

/etc/sysconfig/maldet or /etc/default/maldet have everything commented, so no variables there.

RockyLinux 8

# egrep -r ^default_monitor_mode /usr/local/maldetect*/conf.maldet
/usr/local/maldetect.bk447057/conf.maldet:default_monitor_mode="users"
/usr/local/maldetect/conf.maldet:default_monitor_mode="users"
/usr/local/maldetect.last/conf.maldet:default_monitor_mode="users"

CentOS 7

# egrep -r ^default_monitor_mode /usr/local/maldetect*/conf.maldet
/usr/local/maldetect.bk17268/conf.maldet:default_monitor_mode="users"
/usr/local/maldetect.bk24998/conf.maldet:default_monitor_mode="users"
/usr/local/maldetect/conf.maldet:default_monitor_mode="users"
/usr/local/maldetect.last/conf.maldet:default_monitor_mode="users"

Ubuntu 18

# egrep -r ^default_monitor_mode /usr/local/maldetect*/conf.maldet
/usr/local/maldetect.bk16817/conf.maldet:default_monitor_mode="users"
/usr/local/maldetect.bk1785/conf.maldet:default_monitor_mode="users"
/usr/local/maldetect/conf.maldet:default_monitor_mode="users"
/usr/local/maldetect.last/conf.maldet:default_monitor_mode="users"

Where two of /usr/local/maldetect.bk* are, probably because I've run maldet -d --force in hope will be fixed.

Thank you.

rfxn commented 1 year ago

@nicutor Can you verify you have the 'ed' package installed? which ed

nicutor commented 1 year ago

@rfxn I can confirm that after installing ed on Rocky 8 and CentOS 7 the issue was fixed.

However on Ubuntu 18 ed is already installed and still not work.

# which ed
/bin/ed

# dpkg -L ed
/.
/bin
/bin/red
/bin/ed
...

rfxn commented 1 year ago

Thank you for confirming, update pushed out that makes sure the 'ed' error exposes to the unit file / systemd status. Starting the review on Ubuntu 14 / 18 / 20.

nicutor commented 1 year ago

@rfxn I don't have servers with Ubuntu 20 or 22, but on Ubuntu 18 I was able to start maldet modifying the cpulimit command.

# apt install cpulimit
cpulimit is already the newest version (2.5-1).
# which cpulimit
/usr/bin/cpulimit
# /usr/bin/cpulimit --help
CPUlimit version 2.4

vi /usr/local/maldetect/internals/functions
---
#                       nice_command="$cpulimit -l $scan_cpulimit $nice_command"
                        nice_command="$cpulimit -l $scan_cpulimit -m -- $nice_command"

#                       nice_command="$cpulimit -l $scan_cpulimit -i $nice_command"
                        nice_command="$cpulimit -l $scan_cpulimit -m -- $nice_command"
---

It seems that cpulimit is different on debian versus el systems. Please check it.

rfxn commented 1 year ago

@nicutor Thank you, change pushed up and tested on Ubuntu 18 and 20 enforcing the usage of '--' which is backward compatible and proper usage for all earlier versions of cpulimit.

If you can give maldet an update and test on your Ubuntu system, that would be appreciated! maldet -d --force

nicutor commented 1 year ago

Hi @rfxn Thank you too!

There are 2 lines where this must be changed:

https://github.com/rfxn/linux-malware-detect/blob/master/files/internals/functions#L165 https://github.com/rfxn/linux-malware-detect/blob/master/files/internals/functions#L2012

The second one is for inotify monitoring. There is also a -i argument:

nice_command="$cpulimit -l $scan_cpulimit -i $nice_command"

On the Debian/Ubuntu, the -i argument is missing and probably the -m can take his place:

Debian/Ubuntu:

-m, --monitor-forks
watch and throttle child processes of the target process Warning: It is usually a bad idea to use this flag on a shell script. The commands in the script will each spawn a process which will, in turn, spawn more copies of this program to throttle them, bogging down the system.

EL

      -i, --include-children limit also the children processes

rfxn commented 1 year ago

Apologies missed the second cpulimit call for notify. I've removed the forked throttling as that is problematic and we pass the nice_command with CPU limit on just about anything maldet calls such as find, clamscan, inotifywait etc...

Thank you for the feedback.

rfxn / linux-malware-detect

maldet.service cannot be started #395