Closed anxstj closed 2 years ago
What does your /etc/sos/sos.conf
look like? Was this rpm installed from the CentOS repos or some mirror? Can you check rpm -V sos
to see if perhaps it has been modified in some way?
I just spun up a CS 9 VM, and -o
is working as expected for me out of the box:
[root@cs9test ~]# cat /etc/redhat-release
CentOS Stream release 9
[root@cs9test ~]# rpm -q sos
sos-4.3-1.el9.noarch
[root@cs9test ~]# sos report -o kernel --batch
sosreport (version 4.3)
[...]
Setting up archive ...
Setting up plugins ...
Running plugins. Please wait ...
Starting 1/1 kernel [Running: kernel]
Finished running plugins
Creating compressed archive...
Your sosreport has been generated and saved in:
/var/tmp/sosreport-cs9test-2022-04-20-elolbig.tar.xz
Size 1.40MiB
Owner root
sha256 1074f75bf8c87f7a5654b8f746f199b1e1e1dbe97e993e775679a20036364d20
Please send this file to your support representative.
# grep -v '^#' /etc/sos/sos.conf
[global]
[report]
[collect]
[clean]
[plugin_options]
Nothing there except groups and comments.
# rpm -V sos
# rpm -i sos
error: open of sos failed: No such file or directory
bash-5.1# rpm -qi sos
Name : sos
Version : 4.3
Release : 1.el9
Architecture: noarch
Install Date: Wed 20 Apr 2022 12:57:46 PM UTC
Group : Applications/System
Size : 2767262
License : GPLv2+
Signature : RSA/SHA256, Tue 29 Mar 2022 01:40:25 PM UTC, Key ID 05b555b38483c65d
Source RPM : sos-4.3-1.el9.src.rpm
Build Date : Tue 29 Mar 2022 06:09:13 AM UTC
Build Host : ppc64le-01.stream.rdu2.redhat.com
Packager : builder@centos.org
Vendor : CentOS
URL : https://github.com/sosreport/sos
Summary : A set of tools to gather troubleshooting information from a system
Description :
Sos is a set of tools that gathers information about system
hardware and configuration. The information can then be used for
diagnostic purposes and debugging. Sos is commonly used to help
support technicians and developers.
With the following steps:
docker run -it quay.io/centos/centos:stream9 bash
dnf install sos
sos report -o kernel --batch
I get this result:
sosreport (version 4.3)
WARNING: unable to set option for disabled or non-existing plugin (boot).
...
Setting up archive ...
Setting up plugins ...
[plugin:devicemapper] skipped command 'dmsetup info -c': required kmods missing: dm_mod.
[plugin:devicemapper] skipped command 'dmsetup table': required kmods missing: dm_mod.
[plugin:devicemapper] skipped command 'dmsetup status': required kmods missing: dm_mod.
[plugin:devicemapper] skipped command 'dmsetup ls --tree': required kmods missing: dm_mod.
[plugin:devicemapper] skipped command 'dmsetup udevcookies': required kmods missing: dm_mod.
[plugin:devicemapper] skipped command 'dmstats list': required kmods missing: dm_mod.
[plugin:devicemapper] skipped command 'dmstats print --allregions': required kmods missing: dm_mod.
[plugin:firewall_tables] skipped command 'nft list ruleset': required kmods missing: nfnetlink, nf_tables. Use '--allow-system-changes' to enable collection.
[plugin:firewall_tables] skipped command 'iptables -vnxL': required kmods missing: iptable_filter, nf_tables.
[plugin:firewall_tables] skipped command 'ip6tables -vnxL': required kmods missing: nf_tables, ip6table_filter.
[plugin:networking] skipped command 'ip -s macsec show': required kmods missing: macsec. Use '--allow-system-changes' to enable collection.
[plugin:networking] skipped command 'ss -peaonmi': required kmods missing: udp_diag, xsk_diag, inet_diag, netlink_diag, unix_diag, af_packet_diag, tcp_diag. Use '--allow-system-changes' to enable collection.
Running plugins. Please wait ...
Starting 3/45 cgroups [Running: cgroups]
Starting 4/45 crypto [Running: cgroups crypto]
Starting 1/45 anaconda [Running: cgroups crypto anaconda]
Starting 2/45 block [Running: cgroups crypto anaconda block]
Starting 5/45 dbus [Running: cgroups crypto block dbus]
Starting 6/45 devicemapper [Running: cgroups crypto block devicemapper]
Starting 7/45 devices [Running: crypto block devicemapper devices]
Starting 8/45 dnf [Running: crypto block devices dnf]
Starting 9/45 ebpf [Running: crypto block dnf ebpf]
Starting 10/45 filesys [Running: crypto block dnf filesys]
Starting 11/45 firewall_tables [Running: block dnf filesys firewall_tables]
Starting 12/45 hardware [Running: block dnf filesys hardware]
Starting 13/45 host [Running: block dnf hardware host]
Starting 14/45 i18n [Running: block dnf hardware i18n]
Starting 15/45 kernel [Running: block dnf hardware kernel]
Starting 16/45 krb5 [Running: block dnf kernel krb5]
Starting 17/45 ldap [Running: dnf kernel krb5 ldap]
Starting 18/45 libraries [Running: dnf kernel ldap libraries]
Starting 19/45 libvirt [Running: dnf kernel libraries libvirt]
Starting 20/45 login [Running: dnf kernel libvirt login]
Starting 21/45 logrotate [Running: dnf kernel login logrotate]
Starting 22/45 logs [Running: dnf kernel login logs]
Starting 23/45 lvm2 [Running: dnf kernel login lvm2]
Starting 24/45 md [Running: dnf kernel lvm2 md]
Starting 25/45 memory [Running: dnf kernel md memory]
Starting 26/45 multipath [Running: dnf kernel memory multipath]
Starting 27/45 networking [Running: dnf kernel memory networking]
Starting 28/45 openhpi [Running: dnf memory networking openhpi]
Starting 29/45 pam [Running: dnf memory networking pam]
Starting 30/45 pci [Running: dnf networking pam pci]
Starting 31/45 process [Running: dnf networking pci process]
Starting 32/45 processor [Running: dnf networking process processor]
Starting 33/45 python [Running: dnf networking processor python]
Starting 34/45 release [Running: dnf networking processor release]
Starting 35/45 rpm [Running: networking processor release rpm]
Starting 36/45 scsi [Running: networking processor rpm scsi]
Starting 37/45 selinux [Running: networking processor rpm selinux]
Starting 38/45 services [Running: processor rpm selinux services]
Starting 39/45 ssh [Running: processor rpm selinux ssh]
Starting 40/45 system [Running: processor rpm selinux system]
Starting 41/45 systemd [Running: rpm selinux system systemd]
Starting 42/45 sysvipc [Running: rpm system systemd sysvipc]
Starting 43/45 udev [Running: rpm system systemd udev]
Starting 44/45 xfs [Running: rpm system systemd xfs]
Finishing plugins [Running: rpm system systemd]
Starting 45/45 yum [Running: rpm system systemd yum]
Finishing plugins [Running: rpm systemd yum]
Finishing plugins [Running: rpm yum]
Finishing plugins [Running: yum]
Finished running plugins
Creating compressed archive...
Your sosreport has been generated and saved in:
/var/tmp/sosreport-0ba12bd202d1-2022-04-20-qtiaxhw.tar.xz
Size 3.48MiB
Owner root
sha256 a682dddb268affe2ea6099ba5bc93bb4eaa8cacc62cadfa4a1d009437b1d16ad
Please send this file to your support representative.
It's similar to running sos report --batch
.
Can you enable verbose logging and provide the contents of sos_logs/sos.log
(either in a pastebin or attach the whole file here)? sos report -o kernel -vvv --batch
should be fine.
I attached the output from sos report -o kernel -vvv --batch 2>&1> sos_report.verbose.log
. But I couldn't find a sos.log
file.
The log would be in the generated tarball, but what you've provided is fine.
It looks like the cantboot
preset is getting loaded:
[sos.report:setup] using 'cantboot' preset defaults (--all-logs --plugopts boot.all-images=on,rpm.rpmva=on,rpm.rpmdb=on --profiles boot,storage,system --verify)
[sos.report:setup] effective options now: --all-logs --batch --only-plugins kernel --plugopts boot.all-images=on,rpm.rpmva=on,rpm.rpmdb=on --profiles boot,storage,system -vvv --verify
Which is actually extending the effective list of -o
by enabling the profiles listed.
The cantboot
preset is enabled when sos detects that you're running in emergency or rescue mode as reported by systemd.
So, we have a conflict here:
-o
. It's not so much that -o
isn't working, it's that other bits are working as well.@pmoravec @jjansky1 @slashdd - I'm leaning towards the thought that -o
should block presets from being loaded/probed, but perhaps there's another approach that would be better?
I am on torns, as there are a few opposing factors:
sos
than what the tool automatically detectsonly-plugins
config option (when we dont know if it was manually intentionally set by a user, or it is e.g. automatically set option)?cantboot
preset i somehow specific, more important than the others, right? BUT should our options logic honour different presets with different weight? Will not that be more confusing? I.e. when one adds a custom preset similar to cantboot
, should we treat it as the special-preset-we-have-to-strictly-follow, or as a regular preset with options we can overwrite manually?Still I tend to the option "first apply preset, then overwrite it per user/configfile preferences".
Yeah, this one is tricky. I'm starting to think that perhaps in edge cases like this, we should point to the usage of --preset=none
instead of trying to contort the options handling flow more than we already do?
@anxstj if you run sos report -o kernel --preset=none
, does that change the behavior you're seeing?
Yes, running sos report -o kernel --preset=none
works. Only the kernel plugin will be executed.
Just another idea: I execute this in a container without a running systemd. So maybe the rescue or emergency mode detection doesn't work as expected in this case.
Aha! Got it - because systemd isn't present in your container, sos is loading a fallback "allow everything" init system abstraction. This in turn blindly reports that every service queried to it is running, in an effort to not block service-based plugin collections.
I'm thinking our best approach here is to add a method specific to these runlevel-analogous targets rather than checking them as an actual service.
In the meantime, you can either continue using --preset=none
, or if your goal is to actually capture information from the host instead of within the container you'll need to mount the host's /
filesystem within the container and set the HOST
env var to this location. You'll also need to pass the host's namespaces E.G.: docker run -it -v /:/host -e HOST=/host --privileged --ipc=host --net=host --pid=host $image $cmd
.
This usecase is primarily (solely, really) driven by the RH support-tools container for CoreOS collections, so if you have access to the RH registry you can do podman container runlabel RUN registry.redhat.io/rhel8/support-tools
. That will setup the container with the mount as above, and a few other bits, in a more user-friendly manner.
I like to execute only one plugin and it seems that
-o | --only-plugins
should be used for that purpose. ButIt still executes all default plugins. I would expect that
-o | --only-plugins
only enables the listed plugin(s).Disabling each plugin works, so a workaround could look like:
Edit: Version information: