munin-monitoring / munin

Main repository for munin master / node / plugins
http://munin-monitoring.org
Other
1.98k stars 473 forks source link

dynamic graph zoom - no image #1281

Closed BETLOG closed 4 years ago

BETLOG commented 4 years ago

I'm not even sure what information to supply.. but why would this fail? 2020-02-15--19-03-35_betlogbeast_MunindynamicgraphzoomGoogleChrome

Kubuntu 18.04 google-chrome-stable Version 80.0.3987.106 (Official Build) (64-bit)

user@betlognuc:~$ sudo service munin status
● munin.service - LSB: Create /var/run/munin on boot
   Loaded: loaded (/etc/init.d/munin; generated)
   Active: active (exited) since Fri 2020-02-14 11:30:00 AEST; 1 day 7h ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1409 ExecStart=/etc/init.d/munin start (code=exited, status=0/SUCCESS)

Feb 14 11:30:00 betlognuc systemd[1]: Starting LSB: Create /var/run/munin on boot...                                                                                                          
Feb 14 11:30:00 betlognuc systemd[1]: Started LSB: Create /var/run/munin on boot.                                                                                                             
user@betlognuc:~$ sudo service munin-node status
● munin-node.service - Munin Node                                                                                                                                                             
   Loaded: loaded (/lib/systemd/system/munin-node.service; enabled; vendor preset: enabled)                                                                                                   
   Active: active (running) since Sat 2020-02-15 18:18:00 AEST; 49min ago                                                                                                                     
     Docs: man:munin-node(1)                                                                                                                                                                  
           http://munin.readthedocs.org/en/stable-2.0/reference/munin-node.html                                                                                                               
  Process: 21011 ExecStart=/usr/sbin/munin-node $DAEMON_ARGS (code=exited, status=0/SUCCESS)
 Main PID: 21012 (munin-node)
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/munin-node.service
           └─21012 /usr/bin/perl -wT /usr/sbin/munin-node
cat /var/log/munin/munin-update.log

cat /var/log/munin/munin-node.log
Process Backgrounded
2020/02/15-18:18:00 Munin::Node::Server (type Net::Server::Fork) starting! pid(21012)
Resolved [*]:4949 to [::]:4949, IPv6
Not including resolved host [0.0.0.0] IPv4 because it will be handled by [::] IPv6
Binding to TCP port 4949 on host :: with IPv6
Setting gid to "0 0"
2020/02/15-18:20:08 CONNECT TCP Peer: "[::ffff:192.168.188.30]:57914" Local: "[::ffff:192.168.188.20]:4949"
2020/02/15-18:25:09 CONNECT TCP Peer: "[::ffff:192.168.188.30]:57922" Local: "[::ffff:192.168.188.20]:4949"

user@betlognuc:~$ cat /var/log/munin/munin-cgi-graph.log
user@betlognuc:~$ 
user@betlognuc:~$ cat /var/log/munin/munin-limits.log
user@betlognuc:~$ cat /var/log/munin/munin-cgi-html.log
user@betlognuc:~$ cat /var/log/munin/munin-node-configure.log
Jan 22 13:39:59 - Starting munin-node-configure --shell
ln -s '/usr/share/munin/plugins/acpi' '/etc/munin/plugins/acpi'
ln -s '/usr/share/munin/plugins/apache_accesses' '/etc/munin/plugins/apache_accesses'
ln -s '/usr/share/munin/plugins/apache_processes' '/etc/munin/plugins/apache_processes'
ln -s '/usr/share/munin/plugins/apache_volume' '/etc/munin/plugins/apache_volume'
ln -s '/usr/share/munin/plugins/cpu' '/etc/munin/plugins/cpu'
ln -s '/usr/share/munin/plugins/df' '/etc/munin/plugins/df'
ln -s '/usr/share/munin/plugins/df_inode' '/etc/munin/plugins/df_inode'
ln -s '/usr/share/munin/plugins/diskstats' '/etc/munin/plugins/diskstats'
ln -s '/usr/share/munin/plugins/entropy' '/etc/munin/plugins/entropy'
ln -s '/usr/share/munin/plugins/forks' '/etc/munin/plugins/forks'
ln -s '/usr/share/munin/plugins/fw_packets' '/etc/munin/plugins/fw_packets'
ln -s '/usr/share/munin/plugins/http_loadtime' '/etc/munin/plugins/http_loadtime'
ln -s '/usr/share/munin/plugins/if_' '/etc/munin/plugins/if_eno1'
ln -s '/usr/share/munin/plugins/if_err_' '/etc/munin/plugins/if_err_eno1'
ln -s '/usr/share/munin/plugins/interrupts' '/etc/munin/plugins/interrupts'
ln -s '/usr/share/munin/plugins/irqstats' '/etc/munin/plugins/irqstats'
ln -s '/usr/share/munin/plugins/load' '/etc/munin/plugins/load'
ln -s '/usr/share/munin/plugins/memory' '/etc/munin/plugins/memory'
ln -s '/usr/share/munin/plugins/munin_stats' '/etc/munin/plugins/munin_stats'
ln -s '/usr/share/munin/plugins/netstat' '/etc/munin/plugins/netstat'
ln -s '/usr/share/munin/plugins/nfsd' '/etc/munin/plugins/nfsd'
ln -s '/usr/share/munin/plugins/nfsd4' '/etc/munin/plugins/nfsd4'
ln -s '/usr/share/munin/plugins/open_files' '/etc/munin/plugins/open_files'
ln -s '/usr/share/munin/plugins/open_inodes' '/etc/munin/plugins/open_inodes'
ln -s '/usr/share/munin/plugins/proc_pri' '/etc/munin/plugins/proc_pri'
ln -s '/usr/share/munin/plugins/processes' '/etc/munin/plugins/processes'
ln -s '/usr/share/munin/plugins/swap' '/etc/munin/plugins/swap'
ln -s '/usr/share/munin/plugins/threads' '/etc/munin/plugins/threads'
ln -s '/usr/share/munin/plugins/uptime' '/etc/munin/plugins/uptime'
ln -s '/usr/share/munin/plugins/users' '/etc/munin/plugins/users'
ln -s '/usr/share/munin/plugins/vmstat' '/etc/munin/plugins/vmstat'
The following errors were reported by munin-node-configure --shell
# The following plugins caused errors:
# hddtemp_smartctl:
#       Junk printed to stderr
# ip_:
#       Nothing printed to stdout
#       No valid suggestions
# mysql_:
#       Non-zero exit during autoconf (2)
# postfix_mailqueue:
#       Non-zero exit during autoconf (255)
# postgres_autovacuum:
#       Non-zero exit during autoconf (255)
# postgres_bgwriter:
#       Non-zero exit during autoconf (255)
# postgres_cache_:
#       Non-zero exit during autoconf (255)
# postgres_checkpoints:
#       Non-zero exit during autoconf (255)
# postgres_connections_:
#       Non-zero exit during autoconf (255)
# postgres_connections_db:
#       Non-zero exit during autoconf (255)
# postgres_locks_:
#       Non-zero exit during autoconf (255)
# postgres_oldest_prepared_xact_:
#       Non-zero exit during autoconf (255)
# postgres_prepared_xacts_:
#       Non-zero exit during autoconf (255)
# postgres_querylength_:
#       Non-zero exit during autoconf (255)
# postgres_scans_:
#       Non-zero exit during autoconf (255)
# postgres_size_:
#       Non-zero exit during autoconf (255)
# postgres_transactions_:
#       Non-zero exit during autoconf (255)
# postgres_tuples_:
#       Non-zero exit during autoconf (255)
# postgres_users:
#       Non-zero exit during autoconf (255)
# postgres_xlog:
#       Non-zero exit during autoconf (255)
# proc:
#       In family 'auto' but doesn't have 'autoconf' capability
# sendmail_mailqueue:
#       Non-zero exit during autoconf (255)
# sendmail_mailstats:
#       Non-zero exit during autoconf (255)
# sendmail_mailtraffic:
#       Non-zero exit during autoconf (255)
# varnish_:
#       Non-zero exit during autoconf (255)

user@betlognuc:~$ cat /var/log/munin/munin-graph.log
user@betlognuc:~$ cat /var/log/munin/munin-html.log
user@betlognuc:~$ 
sumpfralle commented 4 years ago

Did you use the apache2 configuration supplied by your distribution? Which distribution do you use?

Regarding debugging: the developer console of your browser will show you, which requests are denied by your webserver.

I assume, that this is a configuration issue of your webserver.

BETLOG commented 4 years ago

2019-09-26-raspbian-buster-lite on a pi zero-W I just did a standard apt install repo install. Server version: Apache/2.4.38 (Raspbian)

Chrome console only shows 404's

Admittedly I was tired when I looked last night, but none of the apache logs showed anything that indicated a fault there.

I have a feeling it's an effect of editing /etc/munin/munin.conf (etc) to:

htmldir /var/cache/munin/www

htmldir /var/www/munin But right now I can't figure out why only that image category would fail.

It's probably something obvious, but my mind starts to feel blunt every time I try to figure it out.

pi@pizero:~$ cat /etc/apache2/sites-enabled/munin.conf 
<VirtualHost *:80> 
        ServerAdmin munin@localhost
        ServerName munin.fritz.box
        ServerAlias munin.local
        DocumentRoot /var/www/munin/
        ErrorLog /var/www/munin/apache-error.log 
        CustomLog /var/www/munin/apache-access.log combined
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
pi@pizero:~$ cat /etc/apache2/apache2.conf                                                              
# This is the main Apache server configuration file.  It contains the
# configuration directives that give the server its instructions.
# See http://httpd.apache.org/docs/2.4/ for detailed information about
# the directives and /usr/share/doc/apache2/README.Debian about Debian specific
# hints.
#
#
# Summary of how the Apache 2 configuration works in Debian:
# The Apache 2 web server configuration in Debian is quite different to
# upstream's suggested way to configure the web server. This is because Debian's
# default Apache2 installation attempts to make adding and removing modules,
# virtual hosts, and extra configuration directives as flexible as possible, in
# order to make automating the changes and administering the server as easy as
# possible.

# It is split into several files forming the configuration hierarchy outlined
# below, all located in the /etc/apache2/ directory:
#
#       /etc/apache2/
#       |-- apache2.conf
#       |       `--  ports.conf
#       |-- mods-enabled
#       |       |-- *.load
#       |       `-- *.conf
#       |-- conf-enabled
#       |       `-- *.conf
#       `-- sites-enabled
#               `-- *.conf
#
#
# * apache2.conf is the main configuration file (this file). It puts the pieces
#   together by including all remaining configuration files when starting up the
#   web server.
#
# * ports.conf is always included from the main configuration file. It is
#   supposed to determine listening ports for incoming connections which can be
#   customized anytime.
#
# * Configuration files in the mods-enabled/, conf-enabled/ and sites-enabled/
#   directories contain particular configuration snippets which manage modules,
#   global configuration fragments, or virtual host configurations,
#   respectively.
#
#   They are activated by symlinking available configuration files from their
#   respective *-available/ counterparts. These should be managed by using our
#   helpers a2enmod/a2dismod, a2ensite/a2dissite and a2enconf/a2disconf. See
#   their respective man pages for detailed information.
#
# * The binary is called apache2. Due to the use of environment variables, in
#   the default configuration, apache2 needs to be started/stopped with
#   /etc/init.d/apache2 or apache2ctl. Calling /usr/bin/apache2 directly will not
#   work with the default configuration.

# Global configuration
#

#
# ServerRoot: The top of the directory tree under which the server's
# configuration, error, and log files are kept.
#
# NOTE!  If you intend to place this on an NFS (or otherwise network)
# mounted filesystem then please read the Mutex documentation (available
# at <URL:http://httpd.apache.org/docs/2.4/mod/core.html#mutex>);
# you will save yourself a lot of trouble.
#
# Do NOT add a slash at the end of the directory path.
#
#ServerRoot "/etc/apache2"

#
# The accept serialization lock file MUST BE STORED ON A LOCAL DISK.
#
#Mutex file:${APACHE_LOCK_DIR} default

#
# The directory where shm and other runtime files will be stored.
#

DefaultRuntimeDir ${APACHE_RUN_DIR}

#
# PidFile: The file in which the server should record its process
# identification number when it starts.
# This needs to be set in /etc/apache2/envvars
#
PidFile ${APACHE_PID_FILE}

#
# Timeout: The number of seconds before receives and sends time out.
#
Timeout 300

#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive On

#
# MaxKeepAliveRequests: The maximum number of requests to allow
# during a persistent connection. Set to 0 to allow an unlimited amount.
# We recommend you leave this number high, for maximum performance.
#
MaxKeepAliveRequests 100

#
# KeepAliveTimeout: Number of seconds to wait for the next request from the
# same client on the same connection.
#
KeepAliveTimeout 5

# These need to be set in /etc/apache2/envvars
User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}

#
# HostnameLookups: Log the names of clients or just their IP addresses
# e.g., www.apache.org (on) or 204.62.129.132 (off).
# The default is off because it'd be overall better for the net if people
# had to knowingly turn this feature on, since enabling it means that
# each client request will result in AT LEAST one lookup request to the
# nameserver.
#
HostnameLookups Off

# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
# container, error messages relating to that virtual host will be
# logged here.  If you *do* define an error logfile for a <VirtualHost>
# container, that host's errors will be logged there and not here.
#
ErrorLog ${APACHE_LOG_DIR}/error.log

#
# LogLevel: Control the severity of messages logged to the error_log.
# Available values: trace8, ..., trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the log level for particular modules, e.g.
# "LogLevel info ssl:warn"
#
LogLevel warn

# Include module configuration:
IncludeOptional mods-enabled/*.load
IncludeOptional mods-enabled/*.conf

# Include list of ports to listen on
Include ports.conf

# Sets the default security model of the Apache2 HTTPD server. It does
# not allow access to the root filesystem outside of /usr/share and /var/www.
# The former is used by web applications packaged in Debian,
# the latter may be used for local directories served by the web server. If
# your system is serving content from a sub-directory in /srv you must allow
# access here, or in any related virtual host.
<Directory />
        Options FollowSymLinks
        AllowOverride None
        Require all denied
</Directory>

<Directory /usr/share>
        AllowOverride None
        Require all granted
</Directory>

<Directory /var/www/>
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
</Directory>

<Directory /var/www/munin/>
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
</Directory>

#<Directory /srv/>
#       Options Indexes FollowSymLinks
#       AllowOverride None
#       Require all granted
#</Directory>

# AccessFileName: The name of the file to look for in each directory
# for additional configuration directives.  See also the AllowOverride
# directive.
#
AccessFileName .htaccess

#
# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
#
<FilesMatch "^\.ht">
        Require all denied
</FilesMatch>

#
# The following directives define some format nicknames for use with
# a CustomLog directive.
#
# These deviate from the Common Log Format definitions in that they use %O
# (the actual bytes sent including headers) instead of %b (the size of the
# requested file), because the latter makes it impossible to detect partial
# requests.
#
# Note that the use of %{X-Forwarded-For}i instead of %h is not recommended.
# Use mod_remoteip instead.
#
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

# Include of directories ignores editors' and dpkg's backup files,
# see README.Debian for details.

# Include generic snippets of statements
IncludeOptional conf-enabled/*.conf

# Include the virtual host configurations:
IncludeOptional sites-enabled/*.conf

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
pi@pizero:~$ 
sumpfralle commented 4 years ago

I just did a standard apt install repo install.

I guess, raspbian simply ships the Debian packages. These are subject to automatic testing (just install - no modifications). Thus I assume, that they should just work. But we will find out, which issue plagues you ...

Chrome console only shows 404's

These are the relevant details, I guess. Which URLs failed? Did you see something related to this requests in your apache error log?

BETLOG commented 4 years ago

chrome console

Failed to load resource: the server responded with a status of 404 (Not Found)
apache_accesses-pinpoint=1581904593,1582012593.png:1 
Failed to load resource: the server responded with a status of 404 (Not Found)
/favicon.ico:1 

/var log...


pi@pizero:~$ ls /var/log/apache2
access.log    error.log    error.log.10.gz  error.log.12.gz  error.log.14.gz  error.log.3.gz  error.log.5.gz  error.log.7.gz  error.log.9.gz
access.log.1  error.log.1  error.log.11.gz  error.log.13.gz  error.log.2.gz   error.log.4.gz  error.log.6.gz  error.log.8.gz  other_vhosts_access.log

pi@pizero:~$ cat /var/log/apache2/*.log [Tue Feb 18 00:00:14.655515 2020] [mpm_event:notice] [pid 402:tid 3069636320] AH00489: Apache/2.4.38 (Raspbian) configured -- resuming normal operations [Tue Feb 18 00:00:14.655740 2020] [core:notice] [pid 402:tid 3069636320] AH00094: Command line: '/usr/sbin/apache2'```

sumpfralle commented 4 years ago

apache_accesses-pinpoint=1581904593,1582012593.png

This is probably the crucial request. Sadly the above is just a short visualization of the URL. You need to copy the full URL via the context menu (right click at the item). The delivery of this path is configured via munin's apache configuration file. As I said: this should just work out of the box. Thus we need to find out, what causes the wrong path to be used or which apache configuration entry changed.

Regarding your apache log: it is quite confusing, that the 404 status did not reach your log. Are you sure, that there is no new entry in error.log right after your request? In this case there would be another proxy (or something else) between your browser and your apache webserver. This would be a bit surprising.

BETLOG commented 4 years ago

Thanks for your patience, I'm not completely clueless, but this is a little outside my usual expertise. (wow, chrome makes it tedious to copy console data)

Failed to load resource: the server responded with a status of 404 (Not Found) apache_accesses-pinpoint=1582050393,1582158393.png:1 http://pizero/munin-cgi/munin-cgi-graph/munin/betlognuc/apache_accesses-pinpoint=1582050393,1582158393.png?&lower_limit=&upper_limit=&size_x=800&size_y=400

Failed to load resource: the server responded with a status of 404 (Not Found) /favicon.ico:1 http://pizero/favicon.ico

Oh... my bad, i forgot I ignorantly specified a different location: ...Still, i see no errors.

pi@pizero:~$ cat /etc/apache2/sites-enabled/munin.conf 
<VirtualHost *:80> 
        ServerAdmin munin@localhost
        ServerName munin.fritz.box
        ServerAlias munin.local
        DocumentRoot /var/www/munin/
        ErrorLog /var/www/munin/apache-error.log 
        CustomLog /var/www/munin/apache-access.log combined
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
pi@pizero:~$ tail -f  /var/www/munin/*.log
==> /var/www/munin/apache-access.log <==
192.168.188.20 - - [20/Feb/2020:10:39:15 +1000] "GET /munin/betlognuc/apache_accesses-year.png HTTP/1.1" 304 183 "http://pizero/munin/betlognuc/apache_accesses.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
192.168.188.20 - - [20/Feb/2020:10:39:15 +1000] "GET /munin/betlognuc/apache_accesses-month.png HTTP/1.1" 304 183 "http://pizero/munin/betlognuc/apache_accesses.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
192.168.188.20 - - [20/Feb/2020:10:39:16 +1000] "GET /static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1582051030&stop_epoch=1582159030 HTTP/1.1" 200 2711 "http://pizero/munin/betlognuc/apache_accesses.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
192.168.188.20 - - [20/Feb/2020:10:39:16 +1000] "GET /static/querystring.js HTTP/1.1" 200 952 "http://pizero/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1582051030&stop_epoch=1582159030" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
192.168.188.20 - - [20/Feb/2020:10:39:16 +1000] "GET /static/formatdate.js HTTP/1.1" 200 4069 "http://pizero/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1582051030&stop_epoch=1582159030" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
192.168.188.20 - - [20/Feb/2020:10:39:16 +1000] "GET /munin-cgi/munin-cgi-graph/munin/betlognuc/apache_accesses-pinpoint=1582051030,1582159030.png?&lower_limit=&upper_limit=&size_x=800&size_y=400 HTTP/1.1" 404 488 "http://pizero/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1582051030&stop_epoch=1582159030" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
192.168.188.20 - - [20/Feb/2020:10:39:32 +1000] "GET /static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1582051030&stop_epoch=1582159030 HTTP/1.1" 200 2712 "http://pizero/munin/betlognuc/apache_accesses.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
192.168.188.20 - - [20/Feb/2020:10:39:32 +1000] "GET /static/querystring.js HTTP/1.1" 200 953 "http://pizero/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1582051030&stop_epoch=1582159030" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
192.168.188.20 - - [20/Feb/2020:10:39:32 +1000] "GET /static/formatdate.js HTTP/1.1" 200 4069 "http://pizero/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1582051030&stop_epoch=1582159030" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
192.168.188.20 - - [20/Feb/2020:10:39:32 +1000] "GET /munin-cgi/munin-cgi-graph/munin/betlognuc/apache_accesses-pinpoint=1582051030,1582159030.png?&lower_limit=&upper_limit=&size_x=800&size_y=400 HTTP/1.1" 404 488 "http://pizero/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1582051030&stop_epoch=1582159030" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"

==> /var/www/munin/apache-error.log <==
^C
pi@pizero:~$ 
BETLOG commented 4 years ago

Sorry for the double post, but I just reset the log path, restarted all the services, did a tail-f on the standard log path, and refreshed the broken page... because it might be clearer or offer perspective.

pi@pizero:~$ sudo nano /etc/apache2/sites-enabled/munin.conf 
...
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
pi@pizero:~$ sudo service munin restart
pi@pizero:~$ sudo service munin-node restart
pi@pizero:~$ sudo service apache2 restart

pi@pizero:~$ tail -f /var/log/apache2/*.log

(and hit refresh in the browser thats already on the broken page)

==> /var/log/apache2/access.log <==

==> /var/log/apache2/error.log <==
[Thu Feb 20 00:00:16.024576 2020] [mpm_event:notice] [pid 402:tid 3069636320] AH00489: Apache/2.4.38 (Raspbian) configured -- resuming normal operations
[Thu Feb 20 00:00:16.024880 2020] [core:notice] [pid 402:tid 3069636320] AH00094: Command line: '/usr/sbin/apache2'
[Thu Feb 20 10:50:06.329288 2020] [mpm_event:notice] [pid 402:tid 3069636320] AH00491: caught SIGTERM, shutting down
[Thu Feb 20 10:50:07.107008 2020] [mpm_event:notice] [pid 25466:tid 3070111456] AH00489: Apache/2.4.38 (Raspbian) configured -- resuming normal operations
[Thu Feb 20 10:50:07.108047 2020] [core:notice] [pid 25466:tid 3070111456] AH00094: Command line: '/usr/sbin/apache2'

==> /var/log/apache2/other_vhosts_access.log <==

==> /var/log/apache2/access.log <==
192.168.188.20 - - [20/Feb/2020:10:50:18 +1000] "GET /static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1582051030&stop_epoch=1582159030 HTTP/1.1" 200 2712 "http://pizero/munin/betlognuc/apache_accesses.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
192.168.188.20 - - [20/Feb/2020:10:50:18 +1000] "GET /static/querystring.js HTTP/1.1" 200 953 "http://pizero/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1582051030&stop_epoch=1582159030" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
192.168.188.20 - - [20/Feb/2020:10:50:18 +1000] "GET /static/formatdate.js HTTP/1.1" 200 4069 "http://pizero/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1582051030&stop_epoch=1582159030" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
192.168.188.20 - - [20/Feb/2020:10:50:18 +1000] "GET /munin-cgi/munin-cgi-graph/munin/betlognuc/apache_accesses-pinpoint=1582051030,1582159030.png?&lower_limit=&upper_limit=&size_x=800&size_y=400 HTTP/1.1" 404 488 "http://pizero/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1582051030&stop_epoch=1582159030" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36"
sumpfralle commented 4 years ago

Thanks for your details.

The following line in the log is the interesting one:

GET /munin-cgi/munin-cgi-graph/munin/betlognuc/apache_accesses-pinpoint=1582051030,1582159030.png?&lower_limit=&upper_limit=&size_x=800&size_y=400 HTTP/1.1" ...

I assume, that you host is named betlognuc. The munin in front of it confuses me a bit: do you use grouping for your hosts (e.g. [munin;betlognuc] in your munin.conf)?

If you do not use this kind of grouping: do you have an idea, where the extra /munin/ in the path comes from?

BETLOG commented 4 years ago

I do: 2020-02-22--09-24-23_betlognuc_pizeroGoogleChrome

pi@pizero:~$ cat /etc/munin/munin.conf
# 
includedir /etc/munin/munin-conf.d
# 
graph_period minute
# 
#htmldir /var/cache/munin/www
htmldir /var/www/munin
max_processes 16
#max_processes 64

[munin;pizero]
    address 127.0.0.1
    use_node_name yes

[munin;fritzbox]
    address 127.0.0.1
    use_node_name yes

[munin;betlognuc]
    address 192.168.188.20
    use_node_name yes

[munin;betlogbeast]
    address 192.168.188.21
    use_node_name yes

and just in case my edits have some bearing on this issue:

pi@pizero:~$ sudo cat /etc/munin/plugin-conf.d/munin-node
# This file is used to configure how the plugins are invoked.
# Place in /etc/munin/plugin-conf.d/ or corresponding directory.
#
# PLEASE NOTE: Changes in the plugin-conf.d directory are only
# read at munin-node startup, so restart at any changes.
#
# user <user>         # Set the user to run the plugin as.
# group <group>       # Set the group to run the plugin as.
# command <command>   # Run <command> instead of the plugin. %c expands to
#                       what would normally be run.
# env.<variable> <value> # Sets <variable> in the plugin's environment, see the
#                       individual plugins to find out which variables they
#                       care about.

[fritzbox_*]
env.fritzbox_ip 192.168.188.1
env.fritzbox_password REMOVED
env.traffic_remove_max true # if you do not want the possible max values
env.host_name fritzbox
env.locale en

[nvidia_gpu_*]
env.smiexec /usr/bin/nvidia-smi
env.warning 75
env.critical 95

[amavis]
group adm
env.MUNIN_MKTEMP /bin/mktemp -p /tmp/ $1
env.amavislog /var/log/mail.info

[apt]
user root

[courier_mta_mailqueue]
group daemon

[courier_mta_mailstats]
group adm

[courier_mta_mailvolume]
group adm

[cps*]
user root

[df*]
env.warning 92
env.critical 98
env.exclude_re ^/run/user

[exim_mailqueue]
group adm, (Debian-exim)

[exim_mailstats]
group adm, (Debian-exim)
env.logdir /var/log/exim4/
env.logname mainlog

[fw_conntrack]
user root

[fw_forwarded_local]
user root

[hddtemp_smartctl]
user root

[hddtemp2]
user root

[if_*]
user root

[if_err_*]
user nobody

[ip_*]
user root

[ipmi_*]
user root

[mysql*]
user root
env.mysqlopts --defaults-file=/etc/mysql/debian.cnf
env.mysqluser root
env.mysqlconnection DBI:mysql:mysql;mysql_read_default_file=/etc/mysql/debian.cnf

[postfix_mailqueue]
user postfix

[postfix_mailstats]
group adm

[postfix_mailvolume]
group adm
env.logfile mail.log

[sendmail_*]
user smmta

[smart_*]
user root

[vlan*]
user root

[ejabberd*]
user ejabberd
env.statuses available away chat xa
env.days 1 7 30

[jmx_*]
env.ip 127.0.0.1
env.port 5400

[samba]
user root

[munin_stats]
user munin
group munin

[postgres_*]
user postgres
env.PGUSER postgres
env.PGPORT 5432

[fail2ban]
user root
sumpfralle commented 4 years ago

OK - thanks.

Could you share the content of munin's apache configuration file with us? Probably (at least on Debian) the file is /etc/munin/apache24.conf. It is symlinked into /etc/apache2/conf-available/.

And (just to be sure that there is nothing else disturbing the configuration) - please run the following:

grep -R munin /etc/apache2
BETLOG commented 4 years ago

/etc/munin/apache24.conf

I'm fairly sure i did not edit that...... and unfortunately it appears that what I was expecting to eventually happen, has happened rather quickly: the sdcard has died. So after I receive the 256MiB sdcards that I just ordered for use as /boot, and set raspbian/munin up again with an ssd or usb as root partition... I'll get back to you on this issue :}

BETLOG commented 4 years ago

Now that I have the new hardware, and have reinstalled raspbian buster lite and munin....and still have the same issue:

Observations from this install that may or may not be related to this issue:

Apologies for the extensive text, but it seemed appropriate to be complete:

pi@pizero:~ $ grep -R munin /etc/apache2
/etc/apache2/sites-available/munin.conf:        ServerAdmin munin@localhost
/etc/apache2/sites-available/munin.conf:        ServerName munin.fritz.box
/etc/apache2/sites-available/munin.conf:        ServerAlias munin.local
/etc/apache2/sites-available/munin.conf:        DocumentRoot /var/cache/munin/www/
/etc/apache2/conf-enabled/munin.conf:# Apache can serve munin's files that are either statically generated
/etc/apache2/conf-enabled/munin.conf:# The relevant munin settings are "html_strategy" and "graph_strategy".
/etc/apache2/conf-enabled/munin.conf:# See /usr/share/doc/munin/README.Debian for details.
/etc/apache2/conf-enabled/munin.conf:ScriptAlias /munin-cgi/munin-cgi-graph /usr/lib/munin/cgi/munin-cgi-graph
/etc/apache2/conf-enabled/munin.conf:Alias /munin/static/ /var/cache/munin/www/static/
/etc/apache2/conf-enabled/munin.conf:<Directory /var/cache/munin/www>
/etc/apache2/conf-enabled/munin.conf:<Directory /usr/lib/munin/cgi>
/etc/apache2/conf-enabled/munin.conf:Alias /munin /var/cache/munin/www
/etc/apache2/conf-enabled/munin.conf:#ScriptAlias /munin /usr/lib/munin/cgi/munin-cgi-html
/etc/apache2/sites-enabled/munin.conf:  ServerAdmin munin@localhost
/etc/apache2/sites-enabled/munin.conf:  ServerName munin.fritz.box
/etc/apache2/sites-enabled/munin.conf:  ServerAlias munin.local
/etc/apache2/sites-enabled/munin.conf:  DocumentRoot /var/cache/munin/www/
/etc/apache2/apache2.conf:<Directory /var/cache/munin/www/>
/etc/apache2/conf-available/munin.conf:# Apache can serve munin's files that are either statically generated
/etc/apache2/conf-available/munin.conf:# The relevant munin settings are "html_strategy" and "graph_strategy".
/etc/apache2/conf-available/munin.conf:# See /usr/share/doc/munin/README.Debian for details.
/etc/apache2/conf-available/munin.conf:ScriptAlias /munin-cgi/munin-cgi-graph /usr/lib/munin/cgi/munin-cgi-graph
/etc/apache2/conf-available/munin.conf:Alias /munin/static/ /var/cache/munin/www/static/
/etc/apache2/conf-available/munin.conf:<Directory /var/cache/munin/www>
/etc/apache2/conf-available/munin.conf:<Directory /usr/lib/munin/cgi>
/etc/apache2/conf-available/munin.conf:Alias /munin /var/cache/munin/www
/etc/apache2/conf-available/munin.conf:#ScriptAlias /munin /usr/lib/munin/cgi/munin-cgi-html
pi@pizero:~ $ cat /etc/apache2/conf-available/munin.conf
# Munin configuration for apache2
#
# Apache can serve munin's files that are either statically generated
# (in a cron job) or generated on demand (via cgi).
# The relevant munin settings are "html_strategy" and "graph_strategy".
# The related apache settings can be adjusted at the end of this file.
#
# See /usr/share/doc/munin/README.Debian for details.

# ***** COMMON SETTINGS FOR ALL STRATEGIES *****

ScriptAlias /munin-cgi/munin-cgi-graph /usr/lib/munin/cgi/munin-cgi-graph
Alias /munin/static/ /var/cache/munin/www/static/

<Directory /var/cache/munin/www>
# THIS CAUSES 403 FORBIDDEN
#    Require local
#    Options None
        Require all granted
        Options FollowSymLinks SymLinksIfOwnerMatch
</Directory>

<Directory /usr/lib/munin/cgi>
    Require local
    <IfModule mod_fcgid.c>
        SetHandler fcgid-script
    </IfModule>
    <IfModule !mod_fcgid.c>
        SetHandler cgi-script
    </IfModule>
</Directory>

# ***** SETTINGS FOR CGI/CRON STRATEGIES *****

# pick _one_ of the following lines depending on your "html_strategy"
# html_strategy: cron (default)
Alias /munin /var/cache/munin/www
# html_strategy: cgi (requires the apache module "cgid" or "fcgid")
#ScriptAlias /munin /usr/lib/munin/cgi/munin-cgi-html
pi@pizero:~ $ cat /etc/apache2/sites-available/munin.conf
<VirtualHost *:80> 
        ServerAdmin munin@localhost
        ServerName munin.fritz.box
        ServerAlias munin.local
        DocumentRoot /var/cache/munin/www/
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
pi@pizero:~ $ cat /etc/apache2/apache2.conf
# This is the main Apache server configuration file.  It contains the
# configuration directives that give the server its instructions.
# See http://httpd.apache.org/docs/2.4/ for detailed information about
# the directives and /usr/share/doc/apache2/README.Debian about Debian specific
# hints.
#
#
# Summary of how the Apache 2 configuration works in Debian:
# The Apache 2 web server configuration in Debian is quite different to
# upstream's suggested way to configure the web server. This is because Debian's
# default Apache2 installation attempts to make adding and removing modules,
# virtual hosts, and extra configuration directives as flexible as possible, in
# order to make automating the changes and administering the server as easy as
# possible.

# It is split into several files forming the configuration hierarchy outlined
# below, all located in the /etc/apache2/ directory:
#
#       /etc/apache2/
#       |-- apache2.conf
#       |       `--  ports.conf
#       |-- mods-enabled
#       |       |-- *.load
#       |       `-- *.conf
#       |-- conf-enabled
#       |       `-- *.conf
#       `-- sites-enabled
#               `-- *.conf
#
#
# * apache2.conf is the main configuration file (this file). It puts the pieces
#   together by including all remaining configuration files when starting up the
#   web server.
#
# * ports.conf is always included from the main configuration file. It is
#   supposed to determine listening ports for incoming connections which can be
#   customized anytime.
#
# * Configuration files in the mods-enabled/, conf-enabled/ and sites-enabled/
#   directories contain particular configuration snippets which manage modules,
#   global configuration fragments, or virtual host configurations,
#   respectively.
#
#   They are activated by symlinking available configuration files from their
#   respective *-available/ counterparts. These should be managed by using our
#   helpers a2enmod/a2dismod, a2ensite/a2dissite and a2enconf/a2disconf. See
#   their respective man pages for detailed information.
#
# * The binary is called apache2. Due to the use of environment variables, in
#   the default configuration, apache2 needs to be started/stopped with
#   /etc/init.d/apache2 or apache2ctl. Calling /usr/bin/apache2 directly will not
#   work with the default configuration.

# Global configuration
#

#
# ServerRoot: The top of the directory tree under which the server's
# configuration, error, and log files are kept.
#
# NOTE!  If you intend to place this on an NFS (or otherwise network)
# mounted filesystem then please read the Mutex documentation (available
# at <URL:http://httpd.apache.org/docs/2.4/mod/core.html#mutex>);
# you will save yourself a lot of trouble.
#
# Do NOT add a slash at the end of the directory path.
#
#ServerRoot "/etc/apache2"

#
# The accept serialization lock file MUST BE STORED ON A LOCAL DISK.
#
#Mutex file:${APACHE_LOCK_DIR} default

#
# The directory where shm and other runtime files will be stored.
#

DefaultRuntimeDir ${APACHE_RUN_DIR}

#
# PidFile: The file in which the server should record its process
# identification number when it starts.
# This needs to be set in /etc/apache2/envvars
#
PidFile ${APACHE_PID_FILE}

#
# Timeout: The number of seconds before receives and sends time out.
#
Timeout 300

#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive On

#
# MaxKeepAliveRequests: The maximum number of requests to allow
# during a persistent connection. Set to 0 to allow an unlimited amount.
# We recommend you leave this number high, for maximum performance.
#
MaxKeepAliveRequests 100

#
# KeepAliveTimeout: Number of seconds to wait for the next request from the
# same client on the same connection.
#
KeepAliveTimeout 5

# These need to be set in /etc/apache2/envvars
User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}

#
# HostnameLookups: Log the names of clients or just their IP addresses
# e.g., www.apache.org (on) or 204.62.129.132 (off).
# The default is off because it'd be overall better for the net if people
# had to knowingly turn this feature on, since enabling it means that
# each client request will result in AT LEAST one lookup request to the
# nameserver.
#
HostnameLookups Off

# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
# container, error messages relating to that virtual host will be
# logged here.  If you *do* define an error logfile for a <VirtualHost>
# container, that host's errors will be logged there and not here.
#
ErrorLog ${APACHE_LOG_DIR}/error.log

#
# LogLevel: Control the severity of messages logged to the error_log.
# Available values: trace8, ..., trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the log level for particular modules, e.g.
# "LogLevel info ssl:warn"
#
LogLevel warn

# Include module configuration:
IncludeOptional mods-enabled/*.load
IncludeOptional mods-enabled/*.conf

# Include list of ports to listen on
Include ports.conf

# Sets the default security model of the Apache2 HTTPD server. It does
# not allow access to the root filesystem outside of /usr/share and /var/www.
# The former is used by web applications packaged in Debian,
# the latter may be used for local directories served by the web server. If
# your system is serving content from a sub-directory in /srv you must allow
# access here, or in any related virtual host.
<Directory />
        Options FollowSymLinks
        AllowOverride None
        Require all denied
</Directory>

<Directory /usr/share>
        AllowOverride None
        Require all granted
</Directory>

<Directory /var/www/>
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
</Directory>

<Directory /var/cache/munin/www/>
        Options Indexes FollowSymLinks
#        AllowOverride None
        AllowOverride All
        Require all granted
</Directory>

#<Directory /srv/>
#       Options Indexes FollowSymLinks
#       AllowOverride None
#       Require all granted
#</Directory>

# AccessFileName: The name of the file to look for in each directory
# for additional configuration directives.  See also the AllowOverride
# directive.
#
AccessFileName .htaccess

#
# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
#
<FilesMatch "^\.ht">
        Require all denied
</FilesMatch>

#
# The following directives define some format nicknames for use with
# a CustomLog directive.
#
# These deviate from the Common Log Format definitions in that they use %O
# (the actual bytes sent including headers) instead of %b (the size of the
# requested file), because the latter makes it impossible to detect partial
# requests.
#
# Note that the use of %{X-Forwarded-For}i instead of %h is not recommended.
# Use mod_remoteip instead.
#
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

# Include of directories ignores editors' and dpkg's backup files,
# see README.Debian for details.

# Include generic snippets of statements
IncludeOptional conf-enabled/*.conf

# Include the virtual host configurations:
IncludeOptional sites-enabled/*.conf

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
pi@pizero:~ $ cat /etc/hosts
127.0.0.1       localhost
127.0.1.1       pizero

::1             localhost ip6-localhost ip6-loopback
ff02::1         ip6-allnodes
ff02::2         ip6-allrouters
pi@pizero:~ $ cat /etc/munin/munin.conf
includedir /etc/munin/munin-conf.d
graph_period minute
htmldir /var/cache/munin/www
max_processes 16

[munin;pizero]
#    address 127.0.0.1
    address 127.0.1.1
    use_node_name no

#[munin;fritzbox]
##    address 127.0.0.1
#    address 127.0.1.1
#    use_node_name no

[munin;betlognuc]
    address 192.168.188.20
    use_node_name no

#[munin;betlogbrick]
#    address 192.168.188.25
#    use_node_name no

[munin;betlogbeast]
    address 192.168.188.21
    use_node_name no
pi@pizero:~ $ cat /etc/munin/munin-node.conf
#
# Example config-file for munin-node
#

log_level 4
log_file /var/log/munin/munin-node.log
pid_file /var/run/munin/munin-node.pid

background 1
setsid 1

user root
group root

# This is the timeout for the whole transaction.
# Units are in sec. Default is 15 min
#
# global_timeout 900

# This is the timeout for each plugin.
# Units are in sec. Default is 1 min
#
# timeout 60

# Regexps for files to ignore
ignore_file [\#~]$
ignore_file DEADJOE$
ignore_file \.bak$
ignore_file %$
ignore_file \.dpkg-(tmp|new|old|dist)$
ignore_file \.rpm(save|new)$
ignore_file \.pod$

# Set this if the client doesn't report the correct hostname when
# telnetting to localhost, port 4949
#
#host_name localhost.localdomain

# A list of addresses that are allowed to connect.  This must be a
# regular expression, since Net::Server does not understand CIDR-style
# network notation unless the perl module Net::CIDR is installed.  You
# may repeat the allow line as many times as you'd like

#allow ^127\.0\.0\.1$
#allow ^::1$
###allow ^::ffff:127\.0\.0\.1$
#
# all - node
allow ^::ffff:127\.0\.0\.1$
allow ^::ffff:127\.0\.1\.1$
# defaults
allow ^127\.0\.0\.1$
allow ^127\.0\.1\.1$
allow ^::1$   
# nuc
#allow ^192\.168\.188\.20$
#allow ^::ffff:192\.168\.188\.20$
# brick
#allow ^192\.168\.188\.25$
#allow ^::ffff:192\.168\.188\.25$
# beast
#allow ^192\.168\.188\.21$
#allow ^::ffff:192\.168\.188\.21$
# pizero - server  
#allow ^192\.168\.188\.30$
#allow ^::ffff:192\.168\.188\.30$

# If you have installed the Net::CIDR perl module, you can use one or more
# cidr_allow and cidr_deny address/mask patterns.  A connecting client must
# match any cidr_allow, and not match any cidr_deny.  Note that a netmask
# *must* be provided, even if it's /32
#
# Example:
#
# cidr_allow 127.0.0.1/32
# cidr_allow 192.0.2.0/24
# cidr_deny  192.0.2.42/32

# Which address to bind to;
host *
# host 127.0.0.1

# And which port
port 4949
pi@pizero:~ $ sudo cat /etc/munin/plugin-conf.d/munin-node
# This file is used to configure how the plugins are invoked.
# Place in /etc/munin/plugin-conf.d/ or corresponding directory.
#
# PLEASE NOTE: Changes in the plugin-conf.d directory are only
# read at munin-node startup, so restart at any changes.
#
# user <user>         # Set the user to run the plugin as.
# group <group>       # Set the group to run the plugin as.
# command <command>   # Run <command> instead of the plugin. %c expands to
#                       what would normally be run.
# env.<variable> <value> # Sets <variable> in the plugin's environment, see the
#                       individual plugins to find out which variables they
#                       care about.

##disabled due to multiple python fail...very tedious because it worked before
#[fritzbox_*]
#env.fritzbox_ip 192.168.188.1
#env.fritzbox_password ---------PASSWORD----------
#env.traffic_remove_max true
#env.host_name fritzbox
##env.host_name fritz
##env.host_name pizero
#env.locale en

[pinger]
timeout 1

[amavis]
group adm
env.MUNIN_MKTEMP /bin/mktemp -p /tmp/ $1
env.amavislog /var/log/mail.info

[apt]
user root

[courier_mta_mailqueue]
group daemon

[courier_mta_mailstats]
group adm

[courier_mta_mailvolume]
group adm

[cps*]
user root

[df*]
env.warning 92
env.critical 98
env.exclude_re ^/run/user

[exim_mailqueue]
group adm, (Debian-exim)

[exim_mailstats]
group adm, (Debian-exim)
env.logdir /var/log/exim4/
env.logname mainlog

[fw_conntrack]
user root

[fw_forwarded_local]
user root

[hddtemp_smartctl]
user root

[hddtemp2]
user root

[if_*]
user root

[if_err_*]
user nobody

[ip_*]
user root

[ipmi_*]
user root

[mysql*]
user root
env.mysqlopts --defaults-file=/etc/mysql/debian.cnf
env.mysqluser root
env.mysqlconnection DBI:mysql:mysql;mysql_read_default_file=/etc/mysql/debian.cnf

[postfix_mailqueue]
user postfix

[postfix_mailstats]
group adm

[postfix_mailvolume]
group adm
env.logfile mail.log

[sendmail_*]
user smmta

[smart_*]
user root

[vlan*]
user root

[ejabberd*]
user ejabberd
env.statuses available away chat xa
env.days 1 7 30

[jmx_*]
env.ip 127.0.0.1
env.port 5400

[samba]
user root

[munin_stats]
user munin
group munin

[postgres_*]
user postgres
env.PGUSER postgres
env.PGPORT 5432

[fail2ban]
user root
pi@pizero:~ $ cat /etc/munin/apache24.conf
# Munin configuration for apache2
#
# Apache can serve munin's files that are either statically generated
# (in a cron job) or generated on demand (via cgi).
# The relevant munin settings are "html_strategy" and "graph_strategy".
# The related apache settings can be adjusted at the end of this file.
#
# See /usr/share/doc/munin/README.Debian for details.

# ***** COMMON SETTINGS FOR ALL STRATEGIES *****

ScriptAlias /munin-cgi/munin-cgi-graph /usr/lib/munin/cgi/munin-cgi-graph
Alias /munin/static/ /var/cache/munin/www/static/

<Directory /var/cache/munin/www>
#    Require local
#    Options None
        Require all granted
        Options FollowSymLinks SymLinksIfOwnerMatch
</Directory>

<Directory /usr/lib/munin/cgi>
    Require local
    <IfModule mod_fcgid.c>
        SetHandler fcgid-script
    </IfModule>
    <IfModule !mod_fcgid.c>
        SetHandler cgi-script
    </IfModule>
</Directory>

# ***** SETTINGS FOR CGI/CRON STRATEGIES *****

# pick _one_ of the following lines depending on your "html_strategy"
# html_strategy: cron (default)
Alias /munin /var/cache/munin/www
# html_strategy: cgi (requires the apache module "cgid" or "fcgid")
#ScriptAlias /munin /usr/lib/munin/cgi/munin-cgi-html
pi@pizero:~ $ sudo service apache2 status
● apache2.service - The Apache HTTP Server
   Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2020-03-13 09:17:36 AEST; 16min ago
     Docs: https://httpd.apache.org/docs/2.4/
  Process: 454 ExecStart=/usr/sbin/apachectl start (code=exited, status=0/SUCCESS)
 Main PID: 486 (apache2)
   Memory: 5.5M
   CGroup: /system.slice/apache2.service
           ├─486 /usr/sbin/apache2 -k start
           ├─487 /usr/sbin/apache2 -k start
           └─488 /usr/sbin/apache2 -k start

Mar 13 09:17:34 pizero systemd[1]: Starting The Apache HTTP Server...
Mar 13 09:17:36 pizero apachectl[454]: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
Mar 13 09:17:36 pizero systemd[1]: Started The Apache HTTP Server.
pi@pizero:~ $ sudo service munin status
● munin.service - LSB: Create munin master directories on boot
   Loaded: loaded (/etc/init.d/munin; generated)
   Active: active (exited) since Fri 2020-03-13 09:17:35 AEST; 16min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 464 ExecStart=/etc/init.d/munin start (code=exited, status=0/SUCCESS)

Mar 13 09:17:34 pizero systemd[1]: Starting LSB: Create munin master directories on boot...
Mar 13 09:17:35 pizero systemd[1]: Started LSB: Create munin master directories on boot.
pi@pizero:~ $ sudo service munin-node status
● munin-node.service - Munin Node
   Loaded: loaded (/lib/systemd/system/munin-node.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2020-03-13 09:17:39 AEST; 16min ago
     Docs: man:munin-node(1)
           http://munin.readthedocs.org/en/stable-2.0/reference/munin-node.html
  Process: 458 ExecStart=/usr/sbin/munin-node $DAEMON_ARGS (code=exited, status=0/SUCCESS)
 Main PID: 651 (munin-node)
   Memory: 11.8M
   CGroup: /system.slice/munin-node.service
           └─651 /usr/bin/perl -wT /usr/sbin/munin-node

Mar 13 09:17:34 pizero systemd[1]: Starting Munin Node...
Mar 13 09:17:39 pizero systemd[1]: Started Munin Node.
pi@pizero:~ $ ls -al /var/log/munin/munin-*.log
-rw-r----- 1 www-data adm       0 Mar 11 22:10 /var/log/munin/munin-cgi-graph.log
-rw-r----- 1 www-data adm       0 Mar 11 22:10 /var/log/munin/munin-cgi-html.log
-rw-r----- 1 munin    adm       0 Mar 13 00:01 /var/log/munin/munin-graph.log
-rw-r----- 1 munin    adm   41952 Mar 13 09:37 /var/log/munin/munin-html.log
-rw-r----- 1 munin    adm   17784 Mar 13 09:35 /var/log/munin/munin-limits.log
-rw-r--r-- 1 root     root   3632 Mar 11 22:09 /var/log/munin/munin-node-configure.log
-rw-r--r-- 1 root     root  55097 Mar 13 09:40 /var/log/munin/munin-node.log
-rw-r----- 1 munin    adm  207797 Mar 13 09:40 /var/log/munin/munin-update.log
pi@pizero:~ $ ls -al /var/log/apache2/*.log
-rw-r----- 1 root adm 278750 Mar 13 09:40 /var/log/apache2/access.log
-rw-r----- 1 root adm    927 Mar 13 09:17 /var/log/apache2/error.log
-rw-r----- 1 root adm      0 Mar 11 22:31 /var/log/apache2/other_vhosts_access.log
pi@pizero:~ $ ls -al /usr/share/munin/plugins
total 1592
drwxr-xr-x 2 root root 12288 Mar 13 09:39 .
drwxr-xr-x 3 root root  4096 Mar 11 22:44 ..
-rwxr-xr-x 1 root root  1983 May 16  2019 acpi
-rwxr-xr-x 1 root root  3244 May 16  2019 amavis
-rwxr-xr-x 1 root root  4987 May 16  2019 apache_accesses
-rwxr-xr-x 1 root root  6370 May 16  2019 apache_processes
-rwxr-xr-x 1 root root  4932 May 16  2019 apache_volume
-rwxr-xr-x 1 root root  2327 May 16  2019 apc_envunit_
-rwxr-xr-x 1 root root  3787 May 16  2019 apc_nis
-rwxr-xr-x 1 root root  4755 May 16  2019 apt
-rwxr-xr-x 1 root root  7189 May 16  2019 apt_all
-rwxr-xr-x 1 root root  3010 May 16  2019 asterisk_channels
-rwxr-xr-x 1 root root  3962 May 16  2019 asterisk_channelstypes
-rwxr-xr-x 1 root root  5669 May 16  2019 asterisk_codecs
-rwxr-xr-x 1 root root  3246 May 16  2019 asterisk_meetme
-rwxr-xr-x 1 root root  3132 May 16  2019 asterisk_meetmeusers
-rwxr-xr-x 1 root root  4880 May 16  2019 asterisk_sipchannels
-rwxr-xr-x 1 root root  4711 May 16  2019 asterisk_sippeers
-rwxr-xr-x 1 root root  3370 May 16  2019 asterisk_voicemail
-rwxr-xr-x 1 root root  2280 May 16  2019 bind9
-rwxr-xr-x 1 root root  6020 May 16  2019 bind9_rndc
-rwxr-xr-x 1 root root  2497 May 16  2019 bonding_err_
-rwxr-xr-x 1 root root  3920 May 16  2019 buddyinfo
-rwxr-xr-x 1 root root  5842 May 16  2019 cmc_tc_sensor_
-rwxr-xr-x 1 root root  2604 May 16  2019 courier_
-rwxr-xr-x 1 root root  1114 May 16  2019 courier_mta_mailqueue
-rwxr-xr-x 1 root root  3920 May 16  2019 courier_mta_mailstats
-rwxr-xr-x 1 root root  3157 May 16  2019 courier_mta_mailvolume
-rwxr-xr-x 1 root root  3931 May 16  2019 cps_
-rwxr-xr-x 1 root root  6922 May 16  2019 cpu
-rwxr-xr-x 1 root root  4699 May 16  2019 cpuspeed
-rwxr-xr-x 1 root root  3821 May 16  2019 cupsys_pages
-rwxr-xr-x 1 root root  4329 May 16  2019 df
-rwxr-xr-x 1 root root  2032 May 16  2019 df_abs
-rwxr-xr-x 1 root root  4380 May 16  2019 df_inode
-rwxr-xr-x 1 root root  6580 May 16  2019 dhcpd3
-rwxr-xr-x 1 root root  3077 May 16  2019 digitemp_
-rwxr-xr-x 1 root root 16882 May 16  2019 diskstat_
-rwxr-xr-x 1 root root 35131 May 16  2019 diskstats
-rwxr-xr-x 1 root root 12059 May 16  2019 ejabberd_
-rwxr-xr-x 1 root root   984 May 16  2019 entropy
-rwxr-xr-x 1 root root  3262 May 16  2019 exim_mailqueue
-rwxr-xr-x 1 root root  1971 May 16  2019 exim_mailqueue_alt
-rwxr-xr-x 1 root root  5638 May 16  2019 exim_mailstats
-rwxr-xr-x 1 root root  2862 May 16  2019 fail2ban
-rwxr-xr-x 1 root root   835 May 16  2019 foldingathome
-rwxr-xr-x 1 root root   907 May 16  2019 foldingathome_rank
-rwxr-xr-x 1 root root   963 May 16  2019 foldingathome_wu
-rwxr-xr-x 1 root root  1027 May 16  2019 forks
-rwxr-xr-x 1 root root  3153 May 16  2019 freeradius_acct
-rwxr-xr-x 1 root root  3638 May 16  2019 freeradius_auth
-rwxr-xr-x 1 root root  3186 May 16  2019 freeradius_proxy_acct
-rwxr-xr-x 1 root root  3684 May 16  2019 freeradius_proxy_auth
-rwxr-xr-x 1 root root  4031 May 16  2019 fw_conntrack
-rwxr-xr-x 1 root root  2024 May 16  2019 fw_forwarded_local
-rwxr-xr-x 1 root root  1758 May 16  2019 fw_packets
-rwxr-xr-x 1 root root  2541 May 16  2019 haproxy_
-rwxr-xr-x 1 root root 12752 May 16  2019 haproxy_ng
-rwxr-xr-x 1 root root   867 May 16  2019 hddtemp
-rwxr-xr-x 1 root root  2338 May 16  2019 hddtemp2
-rwxr-xr-x 1 root root  3087 May 16  2019 hddtempd
-rwxr-xr-x 1 root root 10306 May 16  2019 hddtemp_smartctl
-rwxr-xr-x 1 root root 25200 May 16  2019 hp2000_
-rwxr-xr-x 1 root root  2304 May 16  2019 http_loadtime
-rwxr-xr-x 1 root root  5919 May 16  2019 if_
-rwxr-xr-x 1 root root  3145 May 16  2019 if_err_
-rwxr-xr-x 1 root root  5632 May 16  2019 ifx_concurrent_sessions_
-rwxr-xr-x 1 root root  2034 May 16  2019 interrupts
-rwxr-xr-x 1 root root  5018 May 16  2019 iostat
-rwxr-xr-x 1 root root  5541 May 16  2019 iostat_ios
-rwxr-xr-x 1 root root  4828 May 16  2019 ip_
-rwxr-xr-x 1 root root  1555 May 16  2019 ipac-ng
-rwxr-xr-x 1 root root  3671 May 16  2019 ipmi_
-rwxr-xr-x 1 root root  9806 May 16  2019 ipmi_sensor_
-rwxr-xr-x 1 root root  3084 May 16  2019 ircu
-rwxr-xr-x 1 root root  4291 May 16  2019 irqstats
-rwxr-xr-x 1 root root  2751 May 16  2019 load
-rwxr-xr-x 1 root root  4343 May 16  2019 loggrep
-rwxr-xr-x 1 root root  1854 May 16  2019 lpar_cpu
-rwxr-xr-x 1 root root  3949 May 16  2019 lpstat
-rwxr-xr-x 1 root root  4099 May 16  2019 mailman
-rwxr-xr-x 1 root root  2465 May 16  2019 mailscanner
-rwxr-xr-x 1 root root  2279 May 16  2019 mbmon_
-rwxr-xr-x 1 root root  6067 May 16  2019 memcached_
-rwxr-xr-x 1 root root 67822 May 16  2019 meminfo
-rwxr-xr-x 1 root root 10991 May 16  2019 memory
-rwxr-xr-x 1 root root  5625 May 16  2019 mhttping
-rwxr-xr-x 1 root root  2255 May 16  2019 multiping
-rwxr-xr-x 1 root root  2125 May 16  2019 multips
-rwxr-xr-x 1 root root  3618 May 16  2019 multips_memory
-rwxr-xr-x 1 root root  3843 May 16  2019 munin_stats
-rwxr-xr-x 1 root root  3783 May 16  2019 munin_update
-rwxr-xr-x 1 root root 42371 May 16  2019 mysql_
-rwxr-xr-x 1 root root  1811 May 16  2019 mysql_bytes
-rwxr-xr-x 1 root root  5645 May 16  2019 mysql_innodb
-rwxr-xr-x 1 root root  5790 May 16  2019 mysql_isam_space_
-rwxr-xr-x 1 root root  2643 May 16  2019 mysql_queries
-rwxr-xr-x 1 root root  1534 May 16  2019 mysql_slowqueries
-rwxr-xr-x 1 root root  1619 May 16  2019 mysql_threads
-rwxr-xr-x 1 root root  4367 May 16  2019 named
-rwxr-xr-x 1 root root   812 May 16  2019 netopia
-rwxr-xr-x 1 root root  2680 May 16  2019 netstat
-rwxr-xr-x 1 root root 38945 May 16  2019 netstat_multi
-rwxr-xr-x 1 root root  1265 May 16  2019 nfs4_client
-rwxr-xr-x 1 root root  1122 May 16  2019 nfs_client
-rwxr-xr-x 1 root root  1209 May 16  2019 nfsd
-rwxr-xr-x 1 root root  1442 May 16  2019 nfsd4
-rwxr-xr-x 1 root root  2336 May 16  2019 nginx_request
-rwxr-xr-x 1 root root  3147 May 16  2019 nginx_status
-rwxr-xr-x 1 root root  5176 May 16  2019 nomadix_users_
-rwxr-xr-x 1 root root  5894 May 16  2019 ntp_
-rwxr-xr-x 1 root root  1277 May 16  2019 ntp_kernel_err
-rwxr-xr-x 1 root root  1650 May 16  2019 ntp_kernel_pll_freq
-rwxr-xr-x 1 root root  1255 May 16  2019 ntp_kernel_pll_off
-rwxr-xr-x 1 root root  2163 May 16  2019 ntp_offset
-rwxr-xr-x 1 root root  6578 May 16  2019 ntp_states
-rwxr-xr-x 1 root root  1904 May 16  2019 nut_misc
-rwxr-xr-x 1 root root  2622 May 16  2019 nutups_
-rwxr-xr-x 1 root root  1433 May 16  2019 nut_volts
-rwxr-xr-x 1 root root  5120 May 16  2019 nvidia_
-rwxr-xr-x 1 root root  1225 May 16  2019 open_files
-rwxr-xr-x 1 root root  1077 May 16  2019 open_inodes
-rwxr-xr-x 1 root root  2762 May 16  2019 openvpn
-rwxr-xr-x 1 root root  3597 May 16  2019 perdition
-rwxr-xr-x 1 root root  2654 May 16  2019 pgbouncer_connections
-rwxr-xr-x 1 root root  2394 May 16  2019 pgbouncer_requests
-rwxr-xr-x 1 root root  2698 May 16  2019 ping_
-rwxr-xr-x 1 root root  3543 Mar 12 22:37 pinger
-rw-r--r-- 1 root root  5674 May 16  2019 plugin.sh
-rw-r--r-- 1 root root  4765 May 16  2019 plugins.history
-rwxr-xr-x 1 root root  3498 May 16  2019 pm3users_
-rwxr-xr-x 1 root root  1833 May 16  2019 pop_stats
-rwxr-xr-x 1 root root  4255 May 16  2019 port_
-rwxr-xr-x 1 root root  3105 May 16  2019 postfix_mailqueue
-rwxr-xr-x 1 root root  4627 May 16  2019 postfix_mailstats
-rwxr-xr-x 1 root root  5217 May 16  2019 postfix_mailvolume
-rwxr-xr-x 1 root root  2151 May 16  2019 postgres_autovacuum
-rwxr-xr-x 1 root root  2365 May 16  2019 postgres_bgwriter
-rwxr-xr-x 1 root root  2561 May 16  2019 postgres_cache_
-rwxr-xr-x 1 root root  2168 May 16  2019 postgres_checkpoints
-rwxr-xr-x 1 root root  7453 May 16  2019 postgres_connections_
-rwxr-xr-x 1 root root  2719 May 16  2019 postgres_connections_db
-rwxr-xr-x 1 root root  5228 May 16  2019 postgres_locks_
-rwxr-xr-x 1 root root  2613 May 16  2019 postgres_oldest_prepared_xact_
-rwxr-xr-x 1 root root  2557 May 16  2019 postgres_prepared_xacts_
-rwxr-xr-x 1 root root  3739 May 16  2019 postgres_querylength_
-rwxr-xr-x 1 root root  2311 May 16  2019 postgres_scans_
-rwxr-xr-x 1 root root  2387 May 16  2019 postgres_size_
-rwxr-xr-x 1 root root  3916 May 16  2019 postgres_streaming_
-rwxr-xr-x 1 root root  2763 May 16  2019 postgres_transactions_
-rwxr-xr-x 1 root root  3584 May 16  2019 postgres_tuples_
-rwxr-xr-x 1 root root  2567 May 16  2019 postgres_users
-rwxr-xr-x 1 root root  2706 May 16  2019 postgres_xlog
-rwxr-xr-x 1 root root 22587 May 16  2019 proc
-rwxr-xr-x 1 root root 15620 May 16  2019 processes
-rwxr-xr-x 1 root root  1597 May 16  2019 proc_pri
-rwxr-xr-x 1 root root  2670 May 16  2019 proxy_plugin
-rwxr-xr-x 1 root root  1496 May 16  2019 ps_
-rwxr-xr-x 1 root root  1031 May 16  2019 psu_
-rwxr-xr-x 1 root root  1644 May 16  2019 qmailqstat
-rwxr-xr-x 1 root root  1853 May 16  2019 qmailscan
-rwxr-xr-x 1 root root   754 May 16  2019 qmailscan-simple
-rwxr-xr-x 1 root root  3290 May 16  2019 quota_usage_
-rwxr-xr-x 1 root root  1951 May 16  2019 samba
-rwxr-xr-x 1 root root  3617 May 16  2019 selinux_avcstat
-rwxr-xr-x 1 root root  1373 May 16  2019 sendmail_mailqueue
-rwxr-xr-x 1 root root  1729 May 16  2019 sendmail_mailstats
-rwxr-xr-x 1 root root  1302 May 16  2019 sendmail_mailtraffic
-rwxr-xr-x 1 root root 14441 May 16  2019 sensors_
-rwxr-xr-x 1 root root 10281 May 16  2019 slapd_
-rwxr-xr-x 1 root root  4273 May 16  2019 slapd_bdb_cache_
-rwxr-xr-x 1 root root  2130 May 16  2019 slony_lag_
-rwxr-xr-x 1 root root 26320 May 16  2019 smart_
-rwxr-xr-x 1 root root  2418 May 16  2019 snmp__cpuload
-rwxr-xr-x 1 root root  8137 May 16  2019 snmp__df
-rwxr-xr-x 1 root root  7637 May 16  2019 snmp__df_ram
-rwxr-xr-x 1 root root  4770 May 16  2019 snmp__fc_if_
-rwxr-xr-x 1 root root  3549 May 16  2019 snmp__fc_if_err_
-rwxr-xr-x 1 root root  9216 May 16  2019 snmp__if_
-rwxr-xr-x 1 root root  4802 May 16  2019 snmp__if_err_
-rwxr-xr-x 1 root root 22482 May 16  2019 snmp__if_multi
-rwxr-xr-x 1 root root  2537 May 16  2019 snmp__load
-rwxr-xr-x 1 root root  2702 May 16  2019 snmp__memory
-rwxr-xr-x 1 root root  4849 May 16  2019 snmp__netapp_diskusage_
-rwxr-xr-x 1 root root  3627 May 16  2019 snmp__netapp_inodeusage_
-rwxr-xr-x 1 root root  4020 May 16  2019 snmp__netstat
-rwxr-xr-x 1 root root  2099 May 16  2019 snmp__print_pages
-rwxr-xr-x 1 root root  4011 May 16  2019 snmp__print_supplies
-rwxr-xr-x 1 root root  2202 May 16  2019 snmp__processes
-rwxr-xr-x 1 root root  2593 May 16  2019 snmp__rdp_users
-rwxr-xr-x 1 root root  2955 May 16  2019 snmp__sensors_fsc_bx_fan
-rwxr-xr-x 1 root root  3856 May 16  2019 snmp__sensors_fsc_bx_temp
-rwxr-xr-x 1 root root  3106 May 16  2019 snmp__sensors_fsc_fan
-rwxr-xr-x 1 root root  3343 May 16  2019 snmp__sensors_fsc_temp
-rwxr-xr-x 1 root root  5656 May 16  2019 snmp__sensors_mbm_fan
-rwxr-xr-x 1 root root  5630 May 16  2019 snmp__sensors_mbm_temp
-rwxr-xr-x 1 root root  5565 May 16  2019 snmp__sensors_mbm_volt
-rwxr-xr-x 1 root root  2308 May 16  2019 snmp__swap
-rwxr-xr-x 1 root root  3133 May 16  2019 snmp__uptime
-rwxr-xr-x 1 root root  2255 May 16  2019 snmp__users
-rwxr-xr-x 1 root root  1836 May 16  2019 snmp__winload
-rwxr-xr-x 1 root root  2466 May 16  2019 snmp__winmem
-rwxr-xr-x 1 root root  2130 May 16  2019 snort_alerts
-rwxr-xr-x 1 root root  2156 May 16  2019 snort_bytes_pkt
-rwxr-xr-x 1 root root  2123 May 16  2019 snort_drop_rate
-rwxr-xr-x 1 root root  2253 May 16  2019 snort_pattern_match
-rwxr-xr-x 1 root root  2302 May 16  2019 snort_pkts
-rwxr-xr-x 1 root root  2139 May 16  2019 snort_traffic
-rwxr-xr-x 1 root root  2716 May 16  2019 spamstats
-rwxr-xr-x 1 root root  5969 May 16  2019 squeezebox_
-rwxr-xr-x 1 root root  3779 May 16  2019 squid_cache
-rwxr-xr-x 1 root root  4289 May 16  2019 squid_icp
-rwxr-xr-x 1 root root  3677 May 16  2019 squid_objectsize
-rwxr-xr-x 1 root root  3986 May 16  2019 squid_requests
-rwxr-xr-x 1 root root  4021 May 16  2019 squid_traffic
-rwxr-xr-x 1 root root  2107 May 16  2019 surfboard
-rwxr-xr-x 1 root root  1266 May 16  2019 swap
-rwxr-xr-x 1 root root  3696 May 16  2019 sybase_space
-rwxr-xr-x 1 root root  2161 May 16  2019 tcp
-rwxr-xr-x 1 root root  1425 May 16  2019 threads
-rwxr-xr-x 1 root root 10493 May 16  2019 tomcat_
-rwxr-xr-x 1 root root  3213 May 16  2019 tomcat_access
-rwxr-xr-x 1 root root  3368 May 16  2019 tomcat_jvm
-rwxr-xr-x 1 root root  3688 May 16  2019 tomcat_threads
-rwxr-xr-x 1 root root  3294 May 16  2019 tomcat_volume
-rwxr-xr-x 1 root root   855 May 16  2019 uptime
-rwxr-xr-x 1 root root  2514 May 16  2019 users
-rwxr-xr-x 1 root root 25613 May 16  2019 varnish_
-rwxr-xr-x 1 root root  1722 May 16  2019 vlan_
-rwxr-xr-x 1 root root  6063 May 16  2019 vlan_inetuse_
-rwxr-xr-x 1 root root  2255 May 16  2019 vlan_linkuse_
-rwxr-xr-x 1 root root  1150 May 16  2019 vmstat
-rwxr-xr-x 1 root root  6828 May 16  2019 vserver_cpu_
-rwxr-xr-x 1 root root  3974 May 16  2019 vserver_loadavg
-rwxr-xr-x 1 root root 10090 May 16  2019 vserver_resources
-rwxr-xr-x 1 root root  2184 May 16  2019 yum
-rwxr-xr-x 1 root root 28670 May 16  2019 zimbra_
sumpfralle commented 4 years ago

Looking at this now it occurs to me that munin.fritz.box and munin.local (/etc/apache2/sites-available/munin.conf) do not actually work, I have been opening it by entering pizero/munin in the browser

Regarding munin.fritz.box and munin.local: you specified these as ServerName and ServerAlias in your apache configuration file. I am not sure, whether you use/understand these names properly. The name munin.local is part of the .local top-level-domain. It is usually resolved via mDNS (e.g. avahi). But this only works, if the host announces this name (either being its hostname or being explicitly configured in your mDNS announcement service). The other name (munin.fritz.box) looks like it could be resolved by your router. Are you sure, that you really configured this name there? Since the hostname of that host seems to be pizero, you will need to use something like pizero.fritz.box instead.

I have the feeling, that you misunderstand the apache configuration file as some kind of name resolution service (like DNS). But it is really just a matching property of the apache server: it needs to determine which VirtualHost should be used for an incoming request.

Please make sure, that you only use ServerAlias/ServerName directives, that are really resolved (via DNS) to the IP of your munin host.

Or maybe I just misunderstand your approach.

Strangely I now find that I cannot install fritzbox_* plugins, although I had that running previously. Something about pip vs pip3 and requests/lxml....

pip (or the concept of virtualenv) requires manipulation of environment variables. You could configure this via an EnvironmentFile for the munin-node service, but this approach feels ugly. Regarding your specific problem: simply run apt install python3-requests python3-lxml (most relevant python modules are packaged for Debian and thus also available in Raspbian).

So now my munin install is essentially standard out-of-the-box aside from pinger plugin and the 403-avoiding modification to apache24.conf.

By default the /munin is only accessible locally (from the same host). See /usr/share/doc/munin/README.Debian.gz (Allow non-local access to web interface) for details.

Please describe, what exactly does not work right. And please make sure, that your HTTP requests really reach the right host. You can verify this by taking a look at apache's access log (/var/log/apache2/access.log).

BETLOG commented 4 years ago

Or maybe I just misunderstand your approach.

No, I'm a noob with apache. Frankly I never use those FQDNs anyway, http://pizero/munin is my preference. So I have never noticed this before. Maybe this is slightly less dumb:

pi@pizero:~ $ cat /etc/apache2/sites-available/munin.conf
<VirtualHost *:80> 
        ServerAdmin pi@localhost
        ServerName pizero.fritz.box
        ServerAlias pizero.local
        DocumentRoot /var/cache/munin/www/
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

apt install python3-requests python3-lxml

I was about to say I had already done that... but my .bash_history tells me it's one of the few variants I didn't try. Thanks for stating the obvious, I'll probably discontinue using that plugin anyway, but I'm sure thats the solve.

By default the /munin is only accessible locally (from the same host).

Oh...To me that's an unexpected default. This would seem worthy of a comment section in the relevant config that states this, and offer the pertinent alternatives.

/usr/share/doc/munin/

Does not exist (Kubuntu 18.04) https://www.google.com/search?&q=munin+%22Allow+non-local+access+to+web+interface%22 Is not useful either (i was expecting your source document to be found).

Suggestion/request: There are quite a lot of things about munin that assume prior knowledge. You would probably nullify many questions if you commented-in such things as alternative configurations/short explanations/links to docs. Most of us are probably doing as I did; and just pulling what looks like sensible alternatives from web searches, without really knowing anything except if they end up working, and given how few answers I have found via google results that originate from munin sites/FAQs there seems to be an information disconnect somewhere.

Please describe, what exactly does not work right.

Nothing has changed from the first post, and aside from my overloading you with moot side-information and my own ignorance, everything I have said to describe the issue is still the same. Image fails to load.. dont know why.... any data you need will be forthcoming on request. Sorry for being tedious, I realise queries like this test your patience.

/var/log/apache2/access.log

I'm sure you only need the last few lines, but heres the full load from opening http://pizero/munin, clicking 'day' and successive images through to the 'Dynamically zoomable graph' failure point:

pi@pizero:~ $ tail -f /var/log/apache2/access.log
192.168.188.21 - - [13/Mar/2020:11:09:16 +1000] "GET /munin/munin/pizero/load-week.png HTTP/1.1" 304 183 "http://pizero/munin/munin/pizero/index.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:16 +1000] "GET /munin/munin/pizero/open_inodes-week.png HTTP/1.1" 304 183 "http://pizero/munin/munin/pizero/index.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:16 +1000] "GET /munin/munin/pizero/memory-week.png HTTP/1.1" 304 183 "http://pizero/munin/munin/pizero/index.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:16 +1000] "GET /munin/munin/pizero/users-week.png HTTP/1.1" 200 18705 "http://pizero/munin/munin/pizero/index.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:16 +1000] "GET /munin/munin/pizero/memory-day.png HTTP/1.1" 200 34471 "http://pizero/munin/munin/pizero/index.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:16 +1000] "GET /munin/munin/pizero/swap-week.png HTTP/1.1" 200 19628 "http://pizero/munin/munin/pizero/index.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:16 +1000] "GET /munin/munin/pizero/swap-day.png HTTP/1.1" 200 21408 "http://pizero/munin/munin/pizero/index.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:16 +1000] "GET /munin/munin/pizero/uptime-day.png HTTP/1.1" 200 16898 "http://pizero/munin/munin/pizero/index.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:16 +1000] "GET /munin/munin/pizero/uptime-week.png HTTP/1.1" 200 17671 "http://pizero/munin/munin/pizero/index.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:16 +1000] "GET /munin/static/logo-h.png HTTP/1.1" 200 2081 "http://pizero/munin/static/style-new.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:31 +1000] "GET /munin/ HTTP/1.1" 200 1691 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/comparison-day.html HTTP/1.1" 200 3718 "http://pizero/munin/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/apache_accesses-day.png HTTP/1.1" 200 24178 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/apache_processes-day.png HTTP/1.1" 200 17870 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/diskstats_iops-day.png HTTP/1.1" 200 22959 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/diskstats_iops-day.png HTTP/1.1" 200 27119 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/apache_volume-day.png HTTP/1.1" 200 27364 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/static/style-new.css HTTP/1.1" 200 1638 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/diskstats_latency-day.png HTTP/1.1" 200 29890 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/diskstats_iops-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/df-day.png HTTP/1.1" 200 24342 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/df_inode-day.png HTTP/1.1" 200 19638 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/df-day.png HTTP/1.1" 200 23988 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/static/logo-h.png HTTP/1.1" 304 182 "http://pizero/munin/static/style-new.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/diskstats_throughput-day.png HTTP/1.1" 200 25359 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/df_inode-day.png HTTP/1.1" 200 22523 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/diskstats_throughput-day.png HTTP/1.1" 200 27196 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/diskstats_utilization-day.png HTTP/1.1" 200 18396 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/diskstats_utilization-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/diskstats_latency-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/nvidia_gpu_mem-day.png HTTP/1.1" 200 16474 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/df_inode-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/diskstats_latency-day.png HTTP/1.1" 200 35609 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/fw_packets-day.png HTTP/1.1" 200 19982 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/munin_stats-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/http_loadtime-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/pinger-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/netstat-day.png HTTP/1.1" 200 29279 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/if_err_eno1-day.png HTTP/1.1" 200 19046 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/if_eno1-day.png HTTP/1.1" 200 23536 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/diskstats_utilization-day.png HTTP/1.1" 200 16516 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/if_eno1-day.png HTTP/1.1" 200 22177 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/diskstats_throughput-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/if_err_wlan0-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/df-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/if_wlan0-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/nfs_client-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/fw_packets-day.png HTTP/1.1" 200 24282 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/forks-day.png HTTP/1.1" 200 20176 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/netstat-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/http_loadtime-day.png HTTP/1.1" 200 23147 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/threads-day.png HTTP/1.1" 200 22772 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/threads-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/nfs4_client-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/processes-day.png HTTP/1.1" 200 23446 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/processes-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/proc_pri-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/proc_pri-day.png HTTP/1.1" 200 18728 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/vmstat-day.png HTTP/1.1" 200 24432 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/processes-day.png HTTP/1.1" 200 23467 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/vmstat-day.png HTTP/1.1" 200 25790 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/if_err_eno1-day.png HTTP/1.1" 200 19117 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/forks-day.png HTTP/1.1" 200 23523 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/fw_packets-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/vmstat-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/threads-day.png HTTP/1.1" 200 21359 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/acpi-day.png HTTP/1.1" 200 22420 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/nvidia_gpu_power-day.png HTTP/1.1" 200 23645 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/entropy-day.png HTTP/1.1" 200 19969 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/nvidia_gpu_temp-day.png HTTP/1.1" 200 18941 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/entropy-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/entropy-day.png HTTP/1.1" 200 24474 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/cpu-day.png HTTP/1.1" 200 26278 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/proc_pri-day.png HTTP/1.1" 200 17447 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/acpi-day.png HTTP/1.1" 200 24068 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/cpu-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/open_files-day.png HTTP/1.1" 200 16871 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/acpi-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/forks-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/open_files-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/nvidia_gpu_utilization-day.png HTTP/1.1" 200 25125 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/nvidia_gpu_fan-day.png HTTP/1.1" 200 18448 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/irqstats-day.png HTTP/1.1" 200 97565 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/irqstats-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/open_files-day.png HTTP/1.1" 200 16933 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/irqstats-day.png HTTP/1.1" 200 84087 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/cpu-day.png HTTP/1.1" 200 26175 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/open_inodes-day.png HTTP/1.1" 200 18309 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/open_inodes-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/cpuspeed-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/pizero/interrupts-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/load-day.png HTTP/1.1" 200 20495 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/interrupts-day.png HTTP/1.1" 200 29702 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/pizero/load-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlognuc/load-day.png HTTP/1.1" 200 23937 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/betlogbeast/memory-day.png HTTP/1.1" 200 38729 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/betlognuc/users-day.png HTTP/1.1" 200 17031 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/pizero/users-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/betlogbeast/swap-day.png HTTP/1.1" 200 19445 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/betlognuc/swap-day.png HTTP/1.1" 200 15883 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/pizero/swap-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/betlogbeast/uptime-day.png HTTP/1.1" 200 16470 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/pizero/uptime-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/open_inodes-day.png HTTP/1.1" 200 17298 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/betlogbeast/users-day.png HTTP/1.1" 200 19135 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:56 +1000] "GET /munin/munin/betlogbeast/interrupts-day.png HTTP/1.1" 200 24394 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/betlognuc/uptime-day.png HTTP/1.1" 200 16458 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/pizero/memory-day.png HTTP/1.1" 304 183 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:09:57 +1000] "GET /munin/munin/betlognuc/memory-day.png HTTP/1.1" 200 40007 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:10:07 +1000] "GET /munin/munin/betlognuc/apache_accesses.html HTTP/1.1" 200 2214 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:10:07 +1000] "GET /munin/munin/betlognuc/apache_accesses-week.png HTTP/1.1" 200 17952 "http://pizero/munin/munin/betlognuc/apache_accesses.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:10:07 +1000] "GET /munin/munin/betlognuc/apache_accesses-month.png HTTP/1.1" 200 17389 "http://pizero/munin/munin/betlognuc/apache_accesses.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:10:11 +1000] "GET /munin/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1583953550&stop_epoch=1584061550 HTTP/1.1" 200 2711 "http://pizero/munin/munin/betlognuc/apache_accesses.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:10:11 +1000] "GET /munin/static/querystring.js HTTP/1.1" 200 952 "http://pizero/munin/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1583953550&stop_epoch=1584061550" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:10:11 +1000] "GET /munin/static/formatdate.js HTTP/1.1" 200 4069 "http://pizero/munin/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1583953550&stop_epoch=1584061550" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [13/Mar/2020:11:10:11 +1000] "GET /munin-cgi/munin-cgi-graph/munin/betlognuc/apache_accesses-pinpoint=1583953550,1584061550.png?&lower_limit=&upper_limit=&size_x=800&size_y=400 HTTP/1.1" 403 491 "http://pizero/munin/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1583953550&stop_epoch=1584061550" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
::1 - - [13/Mar/2020:11:10:26 +1000] "GET / HTTP/1.1" 200 11014 "-" "Munin - http_loadtime"
sumpfralle commented 4 years ago

ServerName pizero.fritz.box ServerAlias pizero.local

Since you are using http://pizero/munin to access the service, you should also add ServerAlias pizero. The match is really just literal: the name is equal or it does not match. Maybe you did not notice it for now, since there is only one VirtualHost configured (or you were consistently lucky to reach the right one).

By default the /munin is only accessible locally (from the same host).

Oh...To me that's an unexpected default.

I guess, the expectations were drifting a bit over the last years/decades (from security by default to convenience by default). Anyway: I plan to add a debconf question regarding this setting to Debian's munin packaging. This will turn the surprise into an informed decision.

/usr/share/doc/munin/

Does not exist (Kubuntu 18.04)

Did you really try this on the host, where munin is installed? Debian requires every installed package to have a directory with the same name below /usr/share/doc (at least for the copyright information).

There are quite a lot of things about munin that assume prior knowledge. [..]

Indeed. Thanks for raising awareness. I took some notes.

Sorry for being tedious, I realise queries like this test your patience.

No worries - this journey is also fruitful for me ...

Thanks for your patience, too!

Regarding the failure to load the dynamic graph: I think, you just need to replace also the second occurrence of Require local in /etc/apache2/conf-available/munin.conf with Require all granted. The relevance of this setting really needs to be presented in a more obvious way :(

BETLOG commented 4 years ago

Since you are using http://pizero/munin to access the service, you should also add ServerAlias pizero.

It may just work because the fritzbox (router) does name resolution. Add two ServerAlias, or just replace pizero.local?

I plan to add a debconf question

A noble idea, but I always find myself answering default for these things, and only later looking to the configuration file comments for guidance after I have actually used and thought about it.

Did you really try this on the host, where munin is installed

Ok so I have too many terminals open on too many machines. I have it now. My bad.

Regarding the failure to load the dynamic graph: I think, you just need to replace also the second occurrence of Require local in /etc/apache2/conf-available/munin.conf with Require all granted.

Like this? After this edit, and restarting apache2/munin/node it still does not load the Dynamically zoomable graph image.

<Directory /var/cache/munin/www>
#    Require local
#    Options None
        Require all granted
        Options FollowSymLinks SymLinksIfOwnerMatch
</Directory>

<Directory /usr/lib/munin/cgi>
#    Require local
    Require all granted
    <IfModule mod_fcgid.c>
        SetHandler fcgid-script
    </IfModule>
    <IfModule !mod_fcgid.c>
        SetHandler cgi-script
    </IfModule>
</Directory>
sumpfralle commented 4 years ago

It may just work because the fritzbox (router) does name resolution.

Yes, this is the first step. But the second step is that the apache webserver picks the right VirtualHost section. In your case there is probably no alternative VirtualHost section - thus it works now by accident.

Add two ServerAlias, or just replace pizero.local?

Personally I prefer to configure only the names that I really use. But this is a matter of taste.

After this edit, and restarting apache2/munin/node it still does not load the Dynamically zoomable graph image.

Interesting :(

But now the last few lines in your apache's access log should give the final clue.

BETLOG commented 4 years ago

[edit] restarted all 3 services first

192.168.188.21 - - [14/Mar/2020:11:55:52 +1000] "GET /munin/munin/betlogbeast/proc_pri-day.png HTTP/1.1" 200 18605 "http://pizero/munin/munin/comparison-day.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [14/Mar/2020:11:55:58 +1000] "GET /munin/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1584042653&stop_epoch=1584150653 HTTP/1.1" 200 2712 "http://pizero/munin/munin/betlognuc/apache_accesses.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [14/Mar/2020:11:55:58 +1000] "GET /munin/static/formatdate.js HTTP/1.1" 200 4069 "http://pizero/munin/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1584042653&stop_epoch=1584150653" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [14/Mar/2020:11:55:58 +1000] "GET /munin/static/querystring.js HTTP/1.1" 200 953 "http://pizero/munin/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1584042653&stop_epoch=1584150653" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
192.168.188.21 - - [14/Mar/2020:11:55:58 +1000] "GET /munin-cgi/munin-cgi-graph/munin/betlognuc/apache_accesses-pinpoint=1584042653,1584150653.png?&lower_limit=&upper_limit=&size_x=800&size_y=400 HTTP/1.1" 404 488 "http://pizero/munin/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlognuc/apache_accesses&size_x=800&size_y=400&start_epoch=1584042653&stop_epoch=1584150653" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
sumpfralle commented 4 years ago

I was able to reproduce this behavior here by disabling the cgi module in apache2. Surprisingly this did not lead to warnings issued by apache2 when loading munin's configuration.

Please try a2enmod cgi && service apache2 restart.

I will need to think about a good way to make this kind of misconfiguration more obvious.

sumpfralle commented 4 years ago

Just for the recording: the following addition to the apache2 configuration (within <Directory /usr/lib/munin/cgi>) could be a suitable approach for indicating the source of the misconfiguration (disabled cgi module).

<Files munin-cgi-graph>
    ErrorDocument 404 /link-to-some-helpful-image-explaining-the-problem.png
</Files>
<Files munin-cgi-html>
    ErrorDocument 404 "<h1>Invalid apache2 configuration for Munin</h1><p>Please verify that the apache2 module 'fcgid' or 'cgi' is enabled.</p><p>See <a href=\"http://guide.munin-monitoring.org/en/latest/installation/webserver.html\">munin's documentation</a> for details."
</Files>
BETLOG commented 4 years ago

a2enmod cgi && service apache2 restart

pi@pizero:~ $ sudo su 
root@pizero:/home/pi# a2enmod cgi && service apache2 restart
Your MPM seems to be threaded. Selecting cgid instead of cgi.
Enabling module cgid.
To activate the new configuration, you need to run:
  systemctl restart apache2
root@pizero:/home/pi# systemctl restart apache2
root@pizero:/home/pi# exit
exit

It took a moment to load, initially I thought it was a fail, but it works. apache_accesses-pinpoint... Graph selection image is displayed.

Thanks for that quite extended assistance and fix :)

sumpfralle commented 4 years ago

I am glad, that you setup works now. And I hope that improvements based on our discussion will give future users an easier start. Thanks!

BETLOG commented 11 months ago

After somehow breaking it some time after this post closed, and being without it for a long time, I have recently decided I want dynazoom again and AGAIN attacked the problem. So here's some necropost addendum for seekers on this issue:

I found munin-check kind of useful to see why folder perms were bad. But either things change every time i look away or munin-check itself is a bit weird. I guess it does caveat exactly that, so whatever. Use it anyway, fix what it implies while you watch all the logs and refresh a broken dynazoom page.

Note that 'munin-check' seems to give more consistent results when you do it as root (sudo su), not just using sudo.

Make sure you are seeing system mail:

Date: Sat, 18 Nov 2023 03:32:01 +1000 (AEST)
From: Cron Daemon <root@betlognuc.lan>
To: root@betlognuc.lan
Subject: Cron <www-data@betlognuc> cgitmpdir=$({ cat /etc/munin/munin.conf /etc/munin/munin-conf.d/* 2>/dev/null ||
        true; } | sed -nE 's/^\s*cgitmpdir\s+(\S.*)$/\1/p' | tail -1);
        cgitmpdir=${cgitmpdir:-/var/lib/munin/cgi-tmp}; if [ -d "$cgitmpdir" ]; then find "$cgitmpdir/" -type f
        -mtime +1 -delete; find "$cgitmpdir/" -mindepth 1 -type d -empty -delete; fi

find: cannot delete ‘/var/lib/munin/cgi-tmp/munin-cgi-graph’: Permission denied

Watch the logs: for f in /var/log/munin/munin-*.log /var/log/apache2/*.log; do echo -ne "\\n---- $f ----\\n";tail -n5;done

Maybe make it time specific to filter only immediate-past results for f in /var/log/munin/munin-*.log /var/log/apache2/*.log; do echo -ne "\\n---- $f ----\\n";grep $(date +%H:%M)|tail -n5;done

It's hard to know what needs a restart and what doesnt sometimes... so after each change I'd: servicectl restart munin munin-node apache2

cat -fn 11 /var/log/apache2/error.log [Sat Nov 18 01:21:23.211606 2023] [authz_core:error] [pid 1219537] [client 192.168.1.30:37360] AH01630: client denied by server configuration: /usr/lib/munin/cgi/munin-cgi-graph, referer: http://betlognuc/munin/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=munin/betlogbeast/acpi&size_x=800&size_y=400&start_epoch=1700126140&stop_epoch=1700234140 is fixed by changing: /etc/munin/apache24.conf

<Directory /usr/lib/munin/cgi>
#   Require local
    Require all granted

After this all of the logs are devoid of errors, but dynazoom still didnt work.

following https://serverfault.com/questions/825012/munin-could-not-draw-graph I noticed that /var/lib/munin/cgi-tmp/munin-cgi-graph/ didnt exist...which is weird, i guesss i removed it while on a prior quest to figure it out...

sudo su
mkdir /var/lib/munin/cgi-tmp/munin-cgi-graph/
chown munin:www-data /var/lib/munin/cgi-tmp/munin-cgi-graph/
chmod g+w /var/lib/munin/cgi-tmp/munin-cgi-graph/

and it works, without restarting anything just F5 the page and it loads the dyynazoom image.

NOTE: cgi-tmp is a TEMPORARY directory, so it seems that if there are any folders inside it, even if they have the right permissions, will prevent dynazoom from functioning. As far as I can tell this factor alone has been screwing up my install for years.

rm -rf /var/lib/munin/cgi-tmp/munin-cgi-graph/munin