oxidecomputer / helios

Helios: Or, a Vision in a Dream. A Fragment.
Mozilla Public License 2.0
363 stars 9 forks source link

`/var/adm/messages` does not exist until enabling `system-log:rsyslog` SMF service #22

Open bnaecker opened 2 years ago

bnaecker commented 2 years ago

I've seen in several circumstances that the log file /var/adm/messages does not exist on a Helios system by default. Looking at the SMF services, this seems to correlate with svc:/system/system-log:default being the enabled logging SMF service. I've generally been able to "solve" this by swapping that out for the svc:/system/system-log:rsyslog service. At that point, /var/adm/messages appears, and I see log messages as I'd expect.

I don't have an example system where this is currently the case. However, if I undo the "fix", and re-enable system-log:default, I no longer see messages in /var/adm/messages. I'm either unaware of where those messages are directed if the default flavor is enabled, or there may be an issue with the logging configuration being correctly used to emit such messages.

Here is /etc/syslog.conf on sock.eng.oxide.computer, where I've just done the service-swap song and dance:

#
# CDDL HEADER START
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License, Version 1.0 only
# (the "License").  You may not use this file except in compliance
# with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or http://www.opensolaris.org/os/licensing.
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END
#
#ident  "%Z%%M% %I%     %E% SMI"        /* SunOS 5.0 */
#
# Copyright (c) 1991-1998 by Sun Microsystems, Inc.
# All rights reserved.
#
# syslog configuration file.
#
# This file is processed by m4 so be careful to quote (`') names
# that match m4 reserved words.  Also, within ifdef's, arguments
# containing commas must be quoted.
#
*.err;kern.notice;auth.notice                   /dev/sysmsg
*.err;kern.debug;daemon.notice;mail.crit        /var/adm/messages

*.alert;kern.err;daemon.err                     operator
*.alert                                         root

*.emerg                                         *

# if a non-loghost machine chooses to have authentication messages
# sent to the loghost machine, un-comment out the following line:
#auth.notice                    ifdef(`LOGHOST', /var/log/authlog, @loghost)

mail.debug                      ifdef(`LOGHOST', /var/log/syslog, @loghost)

#
# non-loghost machines will use the following lines to cause "user"
# log messages to be logged locally.
#
ifdef(`LOGHOST', ,
user.err                                        /dev/sysmsg
user.err                                        /var/adm/messages
user.alert                                      `root, operator'
user.emerg                                      *
)
citrus-it commented 1 year ago

I also hit this when testing recovery images. We should just ship an empty file in the images.

citrus-it commented 1 year ago

In the recovery image, this was being caused by a missing file in the image. On a standard helios system, the system-log:default service always touches /var/adm/messages. The recovery image is fixed and I just confirmed that the file is present and full of boot log information.

I don't know what was causing the original reported issue on a standard helios system.

citrus-it commented 1 year ago

I am pretty sure that what bnaecker hit there was https://www.illumos.org/issues/15071 If we're shipping rsyslog with helios, we should adjust the standard syslog manifest as has been done in OmniOS to work around this.