How to use Apache resource by HTTPS in pacemaker?

karippery commented 4 years ago

I have a problem when i change apache url to HTTPS. My company need to use the Apache by HTTPS in pacemaker. I already done SSL config in all servers. Everything working fine until when I add Apache resources in pacemaker. I research a lot and I didn't find any solution for my problem. all example apache resourse (status url ) are in http. please help me And i am working on debian 9

Below link I will explain what i done so far. https://github.com/ClusterLabs/pcs/issues/259

oalbrigt commented 4 years ago

According to the agents metadata it's supposed to only be allowed to be accessed from localhost: https://github.com/ClusterLabs/resource-agents/blob/master/heartbeat/apache#L494-L499

karippery commented 4 years ago

@oalbrigt you mean there is no solution for my problem. I also saw some ask same question on stack https://unix.stackexchange.com/questions/587352/apache-fails-to-load-when-ssl-activated-through-pacemaker.

oalbrigt commented 4 years ago

There is no reason to use HTTPS if you only allow it for localhost.

karippery commented 4 years ago

Sorry i didn't understand. Then why my apache resource is suddenly stopped after i configured SSL (https) apache?

i got this error

* Apache_start_0 on server1 'unknown error' (1): call=18, status=complete, exitreason='Failed to access httpd status page.',
    last-rc-change='Mon Sep 21 12:57:04 2020', queued=0ms, exec=3446ms

this is my apache pcs config

 Resource: Apache (class=ocf provider=heartbeat type=apache)
  Attributes: configfile=/etc/apache2/apache2.conf statusurl=http://localhost/server-status
  Operations: start interval=0s timeout=40s (Apache-start-interval-0s)
              stop interval=0s timeout=60s (Apache-stop-interval-0s)
              monitor interval=1min (Apache-monitor-interval-1min)

How can i start apache resource start again ?

karippery commented 4 years ago

this is my /etc/apache2/sites-available/default-ssl.conf

<IfModule mod_ssl.c>
        <VirtualHost _default_:443>
                ServerAdmin webmaster@localhost

                DocumentRoot /var/www/html

                # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
                # error, crit, alert, emerg.
                # It is also possible to configure the loglevel for particular
                # modules, e.g.
                #LogLevel info ssl:warn

                ErrorLog ${APACHE_LOG_DIR}/error.log
                CustomLog ${APACHE_LOG_DIR}/access.log combined

                # For most configuration files from conf-available/, which are
                # enabled or disabled at a global level, it is possible to
                # include a line for only one particular virtual host. For example the
                # following line enables the CGI configuration for this host only
                # after it has been globally disabled with "a2disconf".
                #Include conf-available/serve-cgi-bin.conf

                #   SSL Engine Switch:
                #   Enable/Disable SSL for this virtual host.
                SSLEngine on

                #   A self-signed (snakeoil) certificate can be created by installing
                #   the ssl-cert package. See
                #   /usr/share/doc/apache2/README.Debian.gz for more info.
                #   If both key and certificate are stored in the same file, only the
                #   SSLCertificateFile directive is needed.
                SSLCertificateFile      /etc/ssl/certs/apache-selfsigned.crt
                SSLCertificateKeyFile /etc/ssl/private/apache-selfsigned.key

                #   Server Certificate Chain:
                #   Point SSLCertificateChainFile at a file containing the
                #   concatenation of PEM encoded CA certificates which form the
                #   certificate chain for the server certificate. Alternatively
                #   the referenced file can be the same as SSLCertificateFile
                #   when the CA certificates are directly appended to the server
                #   certificate for convinience.
                #SSLCertificateChainFile /etc/apache2/ssl.crt/server-ca.crt
              <FilesMatch "\.(cgi|shtml|phtml|php)$">
                                SSLOptions +StdEnvVars
                </FilesMatch>

                <Location /svn>

                   DAV svn
                   SVNParentPath /var/lib/svn

                   AuthType Basic
                   AuthName "Subversion Repository"
                   AuthUserFile /etc/apache2/dav_svn.passwd
                   Require valid-user

                </Location>

              <Directory /var/www/html/redmine>
                RailsBaseURI /redmine
                PassengerResolveSymlinksInDocumentRoot on
              </Directory>

                <Directory /usr/lib/cgi-bin>
                                SSLOptions +StdEnvVars
                </Directory>

        </VirtualHost>
</IfModule>

/etc/apache2/sites-available/000-default.conf

<VirtualHost *:80>
.....
DocumentRoot /var/www/html
Redirect "/" "https://ip_address/"
....
</VirtualHost>

I do the same steps in my 3 server

oalbrigt commented 4 years ago

Add the following to your *:80 section:

<Location /server-status>
 SetHandler server-status
 Order deny,allow
 Deny from all
 Allow from 127.0.0.1
</Location>

karippery commented 4 years ago

unfortunately not working

i add to /etc/apache2/sites-available/000-default.conf

 <Location /server-status>
      SetHandler server-status
      Order deny,allow
      Deny from all
      Allow from 127.0.0.1

  </Location>

And i got this error

Failed Actions:
* Apache_monitor_0 on server3 'unknown error' (1): call=290, status=complete, exitreason='none',
    last-rc-change='Mon Sep 21 14:16:47 2020', queued=0ms, exec=85ms
* Apache_monitor_0 on server1 'unknown error' (1): call=33, status=complete, exitreason='none',
    last-rc-change='Mon Sep 21 14:16:45 2020', queued=0ms, exec=74ms
* Apache_monitor_0 on server2 'unknown error' (1): call=172, status=complete, exitreason='none',
    last-rc-change='Mon Sep 21 14:16:40 2020', queued=0ms, exec=81ms

oalbrigt commented 4 years ago

Try running pcs resource debug-monitor --full Apache to identify where it's failing.

karippery commented 4 years ago

Currently I used separate SSL for each server. Should I use SSL Certificate for Multiple Domains ?

karippery commented 4 years ago

pcs resource debug-monitor --full Apache

Operation monitor for Apache (ocf:heartbeat:apache) returned 7
 >  stderr: + echo
 >  stderr: + printenv
 >  stderr: + sort
 >  stderr: + env=
 >  stderr: AONIX_LM_DIR=/home/TeleUSE/etc
 >  stderr: BXwidgets=/home/BXwidgets
 >  stderr: HA_logfacility=none
 >  stderr: HOME=/root
 >  stderr: LC_ALL=C
 >  stderr: LOGNAME=root
 >  stderr: MAIL=/var/mail/root
 >  stderr: OCF_EXIT_REASON_PREFIX=ocf-exit-reason:
 >  stderr: OCF_RA_VERSION_MAJOR=1
 >  stderr: OCF_RA_VERSION_MINOR=0
 >  stderr: OCF_RESKEY_CRM_meta_class=ocf
 >  stderr: OCF_RESKEY_CRM_meta_id=Apache
 >  stderr: OCF_RESKEY_CRM_meta_migration_threshold=5
 >  stderr: OCF_RESKEY_CRM_meta_provider=heartbeat
 >  stderr: OCF_RESKEY_CRM_meta_resource_stickiness=10
 >  stderr: OCF_RESKEY_CRM_meta_type=apache
 >  stderr: OCF_RESKEY_configfile=/etc/apache2/apache2.conf
 >  stderr: OCF_RESKEY_statusurl=http://localhost/server-status
 >  stderr: OCF_RESOURCE_INSTANCE=Apache
 >  stderr: OCF_RESOURCE_PROVIDER=heartbeat
 >  stderr: OCF_RESOURCE_TYPE=apache
 >  stderr: OCF_ROOT=/usr/lib/ocf
 >  stderr: OCF_TRACE_RA=1
 >  stderr: PATH=/root/.rbenv/shims:/root/.rbenv/bin:/root/.rbenv/shims:/root/.rbenv/bin:/usr/local/bin:/home/TeleUSE/bin:/home/xrt/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/ucb
 >  stderr: PCMK_logfacility=none
 >  stderr: PCMK_service=crm_resource
 >  stderr: PWD=/root
 >  stderr: RBENV_SHELL=bash
 >  stderr: SHELL=/bin/bash
 >  stderr: SHLVL=1
 >  stderr: SSH_CLIENT=10.12.116.46 63097 22
 >  stderr: SSH_CONNECTION=10.12.116.46 63097 10.226.179.205 22
 >  stderr: SSH_TTY=/dev/pts/0
 >  stderr: TERM=xterm
 >  stderr: TeleUSE=/home/TeleUSE
 >  stderr: USER=root
 >  stderr: _=/usr/sbin/pcs
 >  stderr: __OCF_TRC_DEST=
 >  stderr: __OCF_TRC_MANAGE=
 >  stderr: + ocf_is_true
 >  stderr: + false
 >  stderr: + . /usr/lib/ocf/lib/heartbeat/apache-conf.sh
 >  stderr: + . /usr/lib/ocf/lib/heartbeat/http-mon.sh
 >  stderr: + bind_address=127.0.0.1
 >  stderr: + curl_ipv6_opts=
 >  stderr: + ocf_is_true
 >  stderr: + false
 >  stderr: + echo
 >  stderr: + grep -qs ::
 >  stderr: + WGETOPTS=-O- -q -L --no-proxy --bind-address=127.0.0.1
 >  stderr: + CURLOPTS=-o - -Ss -L --interface lo
 >  stderr: + HA_VARRUNDIR=/var/run
 >  stderr: + IBMHTTPD=/opt/IBMHTTPServer/bin/httpd
 >  stderr: + HTTPDLIST=/sbin/httpd2 /usr/sbin/httpd2 /usr/sbin/apache2 /sbin/httpd /usr/sbin/httpd /usr/sbin/apache /opt/IBMHTTPServer/bin/httpd
 >  stderr: + MPM=/usr/share/apache2/find_mpm
 >  stderr: + [ -x /usr/share/apache2/find_mpm ]
 >  stderr: + LOCALHOST=http://localhost
 >  stderr: + HTTPDOPTS=-DSTATUS
 >  stderr: + DEFAULT_IBMCONFIG=/opt/IBMHTTPServer/conf/httpd.conf
 >  stderr: + DEFAULT_SUSECONFIG=/etc/apache2/httpd.conf
 >  stderr: + DEFAULT_RHELCONFIG=/etc/httpd/conf/httpd.conf
 >  stderr: + DEFAULT_DEBIANCONFIG=/etc/apache2/apache2.conf
 >  stderr: + basename /usr/lib/ocf/resource.d/heartbeat/apache
 >  stderr: + CMD=apache
 >  stderr: + OCF_REQUIRED_PARAMS=
 >  stderr: + OCF_REQUIRED_BINARIES=
 >  stderr: + ocf_rarun monitor
 >  stderr: + mk_action_func
 >  stderr: + echo apache_monitor
 >  stderr: + tr - _
 >  stderr: + ACTION_FUNC=apache_monitor
 >  stderr: + validate_args
 >  stderr: + is_function apache_monitor
 >  stderr: + command -v apache_monitor
 >  stderr: + test zapache_monitor = zapache_monitor
 >  stderr: + simple_actions
 >  stderr: + check_required_params
 >  stderr: + local v
 >  stderr: + run_function apache_getconfig
 >  stderr: + is_function apache_getconfig
 >  stderr: + command -v apache_getconfig
 >  stderr: + test zapache_getconfig = zapache_getconfig
 >  stderr: + apache_getconfig
 >  stderr: + HTTPD=
 >  stderr: + PORT=
 >  stderr: + STATUSURL=http://localhost/server-status
 >  stderr: + CONFIGFILE=/etc/apache2/apache2.conf
 >  stderr: + OPTIONS=
 >  stderr: + CLIENT=
 >  stderr: + TESTREGEX=</ *html *>
 >  stderr: + TESTURL=
 >  stderr: + TESTREGEX10=
 >  stderr: + TESTCONFFILE=
 >  stderr: + TESTNAME=
 >  stderr: + : /etc/apache2/envvars
 >  stderr: + source_envfiles /etc/apache2/envvars
 >  stderr: + [ -f /etc/apache2/envvars -a -r /etc/apache2/envvars ]
 >  stderr: + . /etc/apache2/envvars
 >  stderr: + unset HOME
 >  stderr: + [  !=  ]
 >  stderr: + SUFFIX=
 >  stderr: + export APACHE_RUN_USER=www-data
 >  stderr: + export APACHE_RUN_GROUP=www-data
 >  stderr: + export APACHE_PID_FILE=/var/run/apache2/apache2.pid
 >  stderr: + export APACHE_RUN_DIR=/var/run/apache2
 >  stderr: + export APACHE_LOCK_DIR=/var/lock/apache2
 >  stderr: + export APACHE_LOG_DIR=/var/log/apache2
 >  stderr: + export LANG=C
 >  stderr: + export LANG
 >  stderr: + [ X = X -o ! -f  -o ! -x  ]
 >  stderr: + find_httpd_prog
 >  stderr: + HTTPD=
 >  stderr: + [ -f /sbin/httpd2 -a -x /sbin/httpd2 ]
 >  stderr: + [ -f /usr/sbin/httpd2 -a -x /usr/sbin/httpd2 ]
 >  stderr: + [ -f /usr/sbin/apache2 -a -x /usr/sbin/apache2 ]
 >  stderr: + HTTPD=/usr/sbin/apache2
 >  stderr: + break
 >  stderr: + [ X != X -a X/usr/sbin/apache2 != X ]
 >  stderr: + detect_default_config
 >  stderr: + [ -f /etc/apache2/httpd.conf ]
 >  stderr: + [ -f /etc/apache2/apache2.conf ]
 >  stderr: + echo /etc/apache2/apache2.conf
 >  stderr: + DefaultConfig=/etc/apache2/apache2.conf
 >  stderr: + CONFIGFILE=/etc/apache2/apache2.conf
 >  stderr: + [ -n /usr/sbin/apache2 ]
 >  stderr: + basename /usr/sbin/apache2
 >  stderr: + httpd_basename=apache2
 >  stderr: + GetParams /etc/apache2/apache2.conf
 >  stderr: + ConfigFile=/etc/apache2/apache2.conf
 >  stderr: + [ ! -f /etc/apache2/apache2.conf ]
 >  stderr: + get_apache_params /etc/apache2/apache2.conf ServerRoot PidFile Port Listen
 >  stderr: + configfile=/etc/apache2/apache2.conf
 >  stderr: + shift 1
 >  stderr: + echo ServerRoot PidFile Port Listen
 >  stderr: + sed s/ /,/g
 >  stderr: + vars=ServerRoot,PidFile,Port,Listen
 >  stderr: + apachecat /etc/apache2/apache2.conf
 >  stderr: + awk -v vars=ServerRoot,PidFile,Port,Listen
 >  stderr:     BEGIN{
 >  stderr:             split(vars,v,",");
 >  stderr:             for( i in v )
 >  stderr:                     vl[i]=tolower(v[i]);
 >  stderr:     }
 >  stderr:     {
 >  stderr:             for( i in v )
 >  stderr:                     if( tolower($1)==vl[i] ) {
 >  stderr:                     print v[i]"="$2
 >  stderr:                     delete vl[i]
 >  stderr:                     break
 >  stderr:             }
 >  stderr:     }
 >  stderr:
 >  stderr: + awk
 >  stderr:     function procline() {
 >  stderr:             split($0,a);
 >  stderr:             if( a[1]~/^[Ii]nclude$/ ) {
 >  stderr:                     includedir=a[2];
 >  stderr:                     gsub("\"","",includedir);
 >  stderr:                     procinclude(includedir);
 >  stderr:             } else {
 >  stderr:                     if( a[1]=="ServerRoot" ) {
 >  stderr:                             rootdir=a[2];
 >  stderr:                             gsub("\"","",rootdir);
 >  stderr:                     }
 >  stderr:                     print;
 >  stderr:             }
 >  stderr:     }
 >  stderr:     function printfile(infile, a) {
 >  stderr:             while( (getline<infile) > 0 ) {
 >  stderr:                     procline();
 >  stderr:             }
 >  stderr:             close(infile);
 >  stderr:     }
 >  stderr:     function allfiles(dir, cmd,f) {
 >  stderr:             cmd="find -L "dir" -type f";
 >  stderr:             while( ( cmd | getline f ) > 0 ) {
 >  stderr:                     printfile(f);
 >  stderr:             }
 >  stderr:             close(cmd);
 >  stderr:     }
 >  stderr:     function listfiles(pattern, cmd,f) {
 >  stderr:             cmd="ls "pattern" 2>/dev/null";
 >  stderr:             while( ( cmd | getline f ) > 0 ) {
 >  stderr:                     printfile(f);
 >  stderr:             }
 >  stderr:             close(cmd);
 >  stderr:     }
 >  stderr:     function procinclude(spec) {
 >  stderr:             if( rootdir!="" && spec!~/^\// ) {
 >  stderr:                     spec=rootdir"/"spec;
 >  stderr:             }
 >  stderr:             if( isdir(spec) ) {
 >  stderr:                     allfiles(spec); # read all files in a directory (and subdirs)
 >  stderr:             } else {
 >  stderr:                     listfiles(spec); # there could be jokers
 >  stderr:             }
 >  stderr:     }
 >  stderr:     function isdir(s) {
 >  stderr:             return !system("test -d \""s"\"");
 >  stderr:     }
 >  stderr:     { procline(); }
 >  stderr:      /etc/apache2/apache2.conf
 >  stderr: + + sed s/#.*//;s/[[:blank:]]*$//;s/^[[:blank:]]*//
 >  stderr: grep -v ^$
 >  stderr: + eval PidFile=${APACHE_PID_FILE}
 >  stderr: + PidFile=/var/run/apache2/apache2.pid
 >  stderr: + CheckPort
 >  stderr: + ocf_is_decimal
 >  stderr: + false
 >  stderr: + CheckPort
 >  stderr: + ocfError performing operation: Argument list too long
_is_decimal
 >  stderr: + false
 >  stderr: + CheckPort 80
 >  stderr: + ocf_is_decimal 80
 >  stderr: + true
 >  stderr: + [ 80 -gt 0 ]
 >  stderr: + PORT=80
 >  stderr: + break
 >  stderr: + echo
 >  stderr: + grep :
 >  stderr: + Listen=localhost:
 >  stderr: + [ Xhttp://localhost/server-status = X ]
 >  stderr: + test /var/run/apache2/apache2.pid
 >  stderr: + return 0
 >  stderr: + validate_env
 >  stderr: + check_required_binaries
 >  stderr: + local v
 >  stderr: + is_function apache_validate_all
 >  stderr: + command -v apache_validate_all
 >  stderr: + test zapache_validate_all = zapache_validate_all
 >  stderr: + local rc
 >  stderr: + LSB_STATUS_STOPPED=3
 >  stderr: + apache_validate_all
 >  stderr: + [ -z /usr/sbin/apache2 ]
 >  stderr: + [ ! -x /usr/sbin/apache2 ]
 >  stderr: + [ ! -f /etc/apache2/apache2.conf ]
 >  stderr: + [ -n  ]
 >  stderr: + [ -n  ]
 >  stderr: + dirname /var/run/apache2/apache2.pid
 >  stderr: + local a
 >  stderr: + local b
 >  stderr: + [ 1 = 1 ]
 >  stderr: + a=/var/run/apache2/apache2.pid
 >  stderr: + [ 1 ]
 >  stderr: + b=/var/run/apache2/apache2.pid
 >  stderr: + [ /var/run/apache2/apache2.pid = /var/run/apache2/apache2.pid ]
 >  stderr: + break
 >  stderr: + b=/var/run/apache2
 >  stderr: + [ -z /var/run/apache2 -o /var/run/apache2/apache2.pid = /var/run/apache2 ]
 >  stderr: + echo /var/run/apache2
 >  stderr: + return 0
 >  stderr: + ocf_mkstatedir root 755 /var/run/apache2
 >  stderr: + local owner
 >  stderr: + local perms
 >  stderr: + local path
 >  stderr: + owner=root
 >  stderr: + perms=755
 >  stderr: + path=/var/run/apache2
 >  stderr: + test -d /var/run/apache2
 >  stderr: + return 0
 >  stderr: + return 0
 >  stderr: + rc=0
 >  stderr: + [ 0 -ne 0 ]
 >  stderr: + ocf_is_probe
 >  stderr: + [ monitor = monitor -a 0 = 0 ]
 >  stderr: + run_probe
 >  stderr: + is_function apache_probe
 >  stderr: + command -v apache_probe
 >  stderr: + test z = zapache_probe
 >  stderr: + shift 1
 >  stderr: + apache_monitor
 >  stderr: + silent_status
 >  stderr: + local pid
 >  stderr: + get_pid
 >  stderr: + [ -f /var/run/apache2/apache2.pid ]
 >  stderr: + false
 >  stderr: + pid=
 >  stderr: + [ -n  ]
 >  stderr: + : No pid file
 >  stderr: + false
 >  stderr: + [ 1 -ne 0 ]
 >  stderr: + ocf_log info apache not running
 >  stderr: + [ 2 -lt 2 ]
 >  stderr: + __OCF_PRIO=info
 >  stderr: + shift
 >  stderr: + __OCF_MSG=apache not running
 >  stderr: + __OCF_PRIO=INFO
 >  stderr: + [ INFO = DEBUG ]
 >  stderr: + ha_log INFO: apache not running
 >  stderr: + __ha_log INFO: apache not running
 >  stderr: + local ignore_stderr=false
 >  stderr: + local loglevel
 >  stderr: + [ xINFO: apache not running = x--ignore-stderr ]
 >  stderr: + [ none =  ]
 >  stderr: + tty
 >  stderr: + [ x = x0 -a x = xdebug ]
 >  stderr: + [ false = true ]
 >  stderr: + [  ]
 >  stderr: + echo INFO: apache not running
 >  stderr: INFO: apache not running
 >  stderr: + return 0
 >  stderr: + return 7

oalbrigt commented 4 years ago

Try pcs resource debug-start --full. 7 means not running, so that makes sense when it has failed to start.

karippery commented 4 years ago

Is this because i add path configfile=/etc/apache2/apache2.conf ? In my OS(debian) I don't have configfile="/etc/httpd/conf/httpd.conf.

oalbrigt commented 4 years ago

It should be able to detect that you're using apache2.conf, so you can probably unset that parameter.

karippery commented 4 years ago

now I try to stop and start apache2 .
I delete Apache resource and create Apache resource again.
restart Apache2

no difference Apache (ocf::heartbeat:apache): Stopped

And I tried again pcs resource debug-monitor --full Apache

Operation monitor for Apache (ocf:heartbeat:apache) returned 1
 >  stderr: + echo
 >  stderr: + printenv
 >  stderr: + sort
 >  stderr: + env=
 >  stderr: AONIX_LM_DIR=/home/TeleUSE/etc
 >  stderr: BXwidgets=/home/BXwidgets
 >  stderr: HA_logfacility=none
 >  stderr: HOME=/root
 >  stderr: LC_ALL=C
 >  stderr: LOGNAME=root
 >  stderr: MAIL=/var/mail/root
 >  stderr: OCF_EXIT_REASON_PREFIX=ocf-exit-reason:
 >  stderr: OCF_RA_VERSION_MAJOR=1
 >  stderr: OCF_RA_VERSION_MINOR=0
 >  stderr: OCF_RESKEY_CRM_meta_class=ocf
 >  stderr: OCF_RESKEY_CRM_meta_id=Apache
 >  stderr: OCF_RESKEY_CRM_meta_migration_threshold=5
 >  stderr: OCF_RESKEY_CRM_meta_provider=heartbeat
 >  stderr: OCF_RESKEY_CRM_meta_resource_stickiness=10
 >  stderr: OCF_RESKEY_CRM_meta_type=apache
 >  stderr: OCF_RESKEY_configfile=/etc/apache2/apache2.conf
 >  stderr: OCF_RESKEY_statusurl=http://localhost/server-status
 >  stderr: OCF_RESOURCE_INSTANCE=Apache
 >  stderr: OCF_RESOURCE_PROVIDER=heartbeat
 >  stderr: OCF_RESOURCE_TYPE=apache
 >  stderr: OCF_ROOT=/usr/lib/ocf
 >  stderr: OCF_TRACE_RA=1
 >  stderr: PATH=/root/.rbenv/shims:/root/.rbenv/bin:/root/.rbenv/shims:/root/.rbenv/bin:/usr/local/bin:/home/TeleUSE/bin:/home/xrt/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/ucb
 >  stderr: PCMK_logfacility=none
 >  stderr: PCMK_service=crm_resource
 >  stderr: PWD=/root
 >  stderr: RBENV_SHELL=bash
 >  stderr: SHELL=/bin/bash
 >  stderr: SHLVL=1
 >  stderr: SSH_CLIENT=10.12.116.46 63097 22
 >  stderr: SSH_CONNECTION=10.12.116.46 63097 10.226.179.205 22
 >  stderr: SSH_TTY=/dev/pts/0
 >  stderr: TERM=xterm
 >  stderr: TeleUSE=/home/TeleUSE
 >  stderr: USER=root
 >  stderr: _=/usr/sbin/pcs
 >  stderr: __OCF_TRC_DEST=
 >  stderr: __OCF_TRC_MANAGE=
 >  stderr: + ocf_is_true
 >  stderr: + false
 >  stderr: + . /usr/lib/ocf/lib/heartbeat/apache-conf.sh
 >  stderr: + . /usr/lib/ocf/lib/heartbeat/http-mon.sh
 >  stderr: + bind_address=127.0.0.1
 >  stderr: + curl_ipv6_opts=
 >  stderr: + ocf_is_true
 >  stderr: + false
 >  stderr: + echo
 >  stderr: + grep -qs ::
 >  stderr: + WGETOPTS=-O- -q -L --no-proxy --bind-address=127.0.0.1
 >  stderr: + CURLOPTS=-o - -Ss -L --interface lo
 >  stderr: + HA_VARRUNDIR=/var/run
 >  stderr: + IBMHTTPD=/opt/IBMHTTPServer/bin/httpd
 >  stderr: + HTTPDLIST=/sbin/httpd2 /usr/sbin/httpd2 /usr/sbin/apache2 /sbin/httpd /usr/sbin/httpd /usr/sbin/apache /opt/IBMHTTPServer/bin/httpd
 >  stderr: + MPM=/usr/share/apache2/find_mpm
 >  stderr: + [ -x /usr/share/apache2/find_mpm ]
 >  stderr: + LOCALHOST=http://localhost
 >  stderr: + HTTPDOPTS=-DSTATUS
 >  stderr: + DEFAULT_IBMCONFIG=/opt/IBMHTTPServer/conf/httpd.conf
 >  stderr: + DEFAULT_SUSECONFIG=/etc/apache2/httpd.conf
 >  stderr: + DEFAULT_RHELCONFIG=/etc/httpd/conf/httpd.conf
 >  stderr: + DEFAULT_DEBIANCONFIG=/etc/apache2/apache2.conf
 >  stderr: + basename /usr/lib/ocf/resource.d/heartbeat/apache
 >  stderr: + CMD=apache
 >  stderr: + OCF_REQUIRED_PARAMS=
 >  stderr: + OCF_REQUIRED_BINARIES=
 >  stderr: + ocf_rarun monitor
 >  stderr: + mk_action_func
 >  stderr: + echo apache_monitor
 >  stderr: + tr - _
 >  stderr: + ACTION_FUNC=apache_monitor
 >  stderr: + validate_args
 >  stderr: + is_function apache_monitor
 >  stderr: + command -v apache_monitor
 >  stderr: + test zapache_monitor = zapache_monitor
 >  stderr: + simple_actions
 >  stderr: + check_required_params
 >  stderr: + local v
 >  stderr: + run_function apache_getconfig
 >  stderr: + is_function apache_getconfig
 >  stderr: + command -v apache_getconfig
 >  stderr: + test zapache_getconfig = zapache_getconfig
 >  stderr: + apache_getconfig
 >  stderr: + HTTPD=
 >  stderr: + PORT=
 >  stderr: + STATUSURL=http://localhost/server-status
 >  stderr: + CONFIGFILE=/etc/apache2/apache2.conf
 >  stderr: + OPTIONS=
 >  stderr: + CLIENT=
 >  stderr: + TESTREGEX=</ *html *>
 >  stderr: + TESTURL=
 >  stderr: + TESTREGEX10=
 >  stderr: + TESTCONFFILE=
 >  stderr: + TESTNAME=
 >  stderr: + : /etc/apache2/envvars
 >  stderr: + source_envfiles /etc/apache2/envvars
 >  stderr: + [ -f /etc/apache2/envvars -a -r /etc/apache2/envvars ]
 >  stderr: + . /etc/apache2/envvars
 >  stderr: + unset HOME
 >  stderr: + [  !=  ]
 >  stderr: + SUFFIX=
 >  stderr: + export APACHE_RUN_USER=www-data
 >  stderr: + export APACHE_RUN_GROUP=www-data
 >  stderr: + export APACHE_PID_FILE=/var/run/apache2/apache2.pid
 >  stderr: + export APACHE_RUN_DIR=/var/run/apache2
 >  stderr: + export APACHE_LOCK_DIR=/var/lock/apache2
 >  stderr: + export APACHE_LOG_DIR=/var/log/apache2
 >  stderr: + export LANG=C
 >  stderr: + export LANG
 >  stderr: + [ X = X -o ! -f  -o ! -x  ]
 >  stderr: + find_httpd_prog
 >  stderr: + HTTPD=
 >  stderr: + [ -f /sbin/httpd2 -a -x /sbin/httpd2 ]
 >  stderr: + [ -f /usr/sbin/httpd2 -a -x /usr/sbin/httpd2 ]
 >  stderr: + [ -f /usr/sbin/apache2 -a -x /usr/sbin/apache2 ]
 >  stderr: + HTTPD=/usr/sbin/apache2
 >  stderr: + break
 >  stderr: + [ X != X -a X/usr/sbin/apache2 != X ]
 >  stderr: + detect_default_config
 >  stderr: + [ -f /etc/apache2/httpd.conf ]
 >  stderr: + [ -f /etc/apache2/apache2.conf ]
 >  stderr: + echo /etc/apache2/apache2.conf
 >  stderr: + DefaultConfig=/etc/apache2/apache2.conf
 >  stderr: + CONFIGFILE=/etc/apache2/apache2.conf
 >  stderr: + [ -n /usr/sbin/apache2 ]
 >  stderr: + basename /usr/sbin/apache2
 >  stderr: + httpd_basename=apache2
 >  stderr: + GetParams /etc/apache2/apache2.conf
 >  stderr: + ConfigFile=/etc/apache2/apache2.conf
 >  stderr: + [ ! -f /etc/apache2/apache2.conf ]
 >  stderr: + get_apache_params /etc/apache2/apache2.conf ServerRoot PidFile Port Listen
 >  stderr: + configfile=/etc/apache2/apache2.conf
 >  stderr: + shift 1
 >  stderr: + echo ServerRoot PidFile Port Listen
 >  stderr: + sed s/ /,/g
 >  stderr: + vars=ServerRoot,PidFile,Port,Listen
 >  stderr: + apachecat /etc/apache2/apache2.conf
 >  stderr: + awk -v vars=ServerRoot,PidFile,Port,Listen
 >  stderr:     BEGIN{
 >  stderr:             split(vars,v,",");
 >  stderr:             for( i in v )
 >  stderr:                     vl[i]=tolower(v[i]);
 >  stderr:     }
 >  stderr:     {
 >  stderr:             for( i in v )
 >  stderr:                     if( tolower($1)==vl[i] ) {
 >  stderr:                     print v[i]"="$2
 >  stderr:                     delete vl[i]
 >  stderr:                     break
 >  stderr:             }
 >  stderr:     }
 >  stderr:
 >  stderr: + awk
 >  stderr:     function procline() {
 >  stderr:             split($0,a);
 >  stderr:             if( a[1]~/^[Ii]nclude$/ ) {
 >  stderr:                     includedir=a[2];
 >  stderr:                     gsub("\"","",includedir);
 >  stderr:                     procinclude(includedir);
 >  stderr:             } else {
 >  stderr:                     if( a[1]=="ServerRoot" ) {
 >  stderr:                             rootdir=a[2];
 >  stderr:                             gsub("\"","",rootdir);
 >  stderr:                     }
 >  stderr:                     print;
 >  stderr:             }
 >  stderr:     }
 >  stderr:     function printfile(infile, a) {
 >  stderr:             while( (getline<infile) > 0 ) {
 >  stderr:                     procline();
 >  stderr:             }
 >  stderr:             close(infile);
 >  stderr:     }
 >  stderr:     function allfiles(dir, cmd,f) {
 >  stderr:             cmd="find -L "dir" -type f";
 >  stderr:             while( ( cmd | getline f ) > 0 ) {
 >  stderr:                     printfile(f);
 >  stderr:             }
 >  stderr:             close(cmd);
 >  stderr:     }
 >  stderr:     function listfiles(pattern, cmd,f) {
 >  stderr:             cmd="ls "pattern" 2>/dev/null";
 >  stderr:             while( ( cmd | getline f ) > 0 ) {
 >  stderr:                     printfile(f);
 >  stderr:             }
 >  stderr:             close(cmd);
 >  stderr:     }
 >  stderr:     function procinclude(spec) {
 >  stderr:             if( rootdir!="" && spec!~/^\// ) {
 >  stderr:                     spec=rootdir"/"spec;
 >  stderr:             }
 >  stderr:             if( isdir(spec) ) {
 >  stderr:                     allfiles(spec); # read all files in a directory (and subdirs)
 >  stderr:             } else {
 >  stderr:                     listfiles(spec); # there could be jokers
 >  stderr:             }
 >  stderr:     }
 >  stderr:     function isdir(s) {
 >  stderr:             return !system("test -d \""s"\"");
 >  stderr:     }
 >  stderr:     { procline(); }
 >  stderr:      /etc/apache2/apache2.conf
 >  stderr: + sed s/#.*//;s/[[:blank:]]*$//;s/^[[:blank:]]*//
 >  stderr: + grep -v ^$
 >  stderr: + eval PidFile=${APACHE_PID_FILE}
 >  stderr: + PidFile=/var/run/apache2/apache2.pid
 >  stderr: + CheckPort
 >  stderr: + ocf_is_decimal
 >  stderr: + false
 >  stderr: + CheckPort
 >  stderr: + ocfError performing operation: Operation not permitted
_is_decimal
 >  stderr: + false
 >  stderr: + CheckPort 80
 >  stderr: + ocf_is_decimal 80
 >  stderr: + true
 >  stderr: + [ 80 -gt 0 ]
 >  stderr: + PORT=80
 >  stderr: + break
 >  stderr: + echo
 >  stderr: + grep :
 >  stderr: + Listen=localhost:
 >  stderr: + [ Xhttp://localhost/server-status = X ]
 >  stderr: + test /var/run/apache2/apache2.pid
 >  stderr: + return 0
 >  stderr: + validate_env
 >  stderr: + check_required_binaries
 >  stderr: + local v
 >  stderr: + is_function apache_validate_all
 >  stderr: + command -v apache_validate_all
 >  stderr: + test zapache_validate_all = zapache_validate_all
 >  stderr: + local rc
 >  stderr: + LSB_STATUS_STOPPED=3
 >  stderr: + apache_validate_all
 >  stderr: + [ -z /usr/sbin/apache2 ]
 >  stderr: + [ ! -x /usr/sbin/apache2 ]
 >  stderr: + [ ! -f /etc/apache2/apache2.conf ]
 >  stderr: + [ -n  ]
 >  stderr: + [ -n  ]
 >  stderr: + dirname /var/run/apache2/apache2.pid
 >  stderr: + local a
 >  stderr: + local b
 >  stderr: + [ 1 = 1 ]
 >  stderr: + a=/var/run/apache2/apache2.pid
 >  stderr: + [ 1 ]
 >  stderr: + b=/var/run/apache2/apache2.pid
 >  stderr: + [ /var/run/apache2/apache2.pid = /var/run/apache2/apache2.pid ]
 >  stderr: + break
 >  stderr: + b=/var/run/apache2
 >  stderr: + [ -z /var/run/apache2 -o /var/run/apache2/apache2.pid = /var/run/apache2 ]
 >  stderr: + echo /var/run/apache2
 >  stderr: + return 0
 >  stderr: + ocf_mkstatedir root 755 /var/run/apache2
 >  stderr: + local owner
 >  stderr: + local perms
 >  stderr: + local path
 >  stderr: + owner=root
 >  stderr: + perms=755
 >  stderr: + path=/var/run/apache2
 >  stderr: + test -d /var/run/apache2
 >  stderr: + return 0
 >  stderr: + return 0
 >  stderr: + rc=0
 >  stderr: + [ 0 -ne 0 ]
 >  stderr: + ocf_is_probe
 >  stderr: + [ monitor = monitor -a 0 = 0 ]
 >  stderr: + run_probe
 >  stderr: + is_function apache_probe
 >  stderr: + command -v apache_probe
 >  stderr: + test z = zapache_probe
 >  stderr: + shift 1
 >  stderr: + apache_monitor
 >  stderr: + silent_status
 >  stderr: + local pid
 >  stderr: + get_pid
 >  stderr: + [ -f /var/run/apache2/apache2.pid ]
 >  stderr: + cat /var/run/apache2/apache2.pid
 >  stderr: + pid=28819
 >  stderr: + [ -n 28819 ]
 >  stderr: + ProcessRunning 28819
 >  stderr: + local pid=28819
 >  stderr: + [ -d /proc -a -d /proc/1 ]
 >  stderr: + [ -d /proc/28819 ]
 >  stderr: + [ 0 -ne 0 ]
 >  stderr: + findhttpclient
 >  stderr: + [ x != x ]
 >  stderr: + which wget
 >  stderr: + echo wget
 >  stderr: + ourhttpclient=wget
 >  stderr: + [ -z wget ]
 >  stderr: + ocf_check_level 10
 >  stderr: + local lvl prev
 >  stderr: + lvl=0
 >  stderr: + prev=0
 >  stderr: + ocf_is_decimal 0
 >  stderr: + true
 >  stderr: + [ 10 -eq 0 ]
 >  stderr: + [ 10 -gt 0 ]
 >  stderr: + lvl=0
 >  stderr: + break
 >  stderr: + echo 0
 >  stderr: + apache_monitor_basic
 >  stderr: + wget_func http://localhost/server-status
 >  stderr: + auth=
 >  stderr: + cl_opts=-O- -q -L --no-proxy --bind-address=127.0.0.1
 >  stderr: + [+  x != x ]
 >  stderr: grep+  -Eiwget </ *html *> -O-
 >  stderr:  -q -L --no-proxy --bind-address=127.0.0.1 http://localhost/server-status
 >  stderr: + attempt_index_monitor_request
 >  stderr: + local indexpage=
 >  stderr: + [ -n  ]
 >  stderr: + [ -n  ]
 >  stderr: + [ -n  ]
 >  stderr: + [ -n http://localhost/server-status ]
 >  stderr: + return 1
 >  stderr: + [ 1 -eq 0 ]
 >  stderr: + ocf_is_probe
 >  stderr: + [ monitor = monitor -a 0 = 0 ]
 >  stderr: + return 1

there is no return 7

karippery commented 4 years ago

Apache is working smoothly from outside of pacemaker. https://virtual_ip/redmine/

The only problem is Apache inside pacemaker (ocf:heartbeat:apache)

redm

oalbrigt commented 4 years ago

Which version of resource-agents do you have installed? I'm unable to find that "monitor" -a line before the final return 1 line.

karippery commented 4 years ago

sorry how should i check resource-agents version?

oalbrigt commented 4 years ago

Also you might have to run a2ensite 000-default to ensure that part of the configuration is loaded.

karippery commented 4 years ago

dpkg -l|grep -i pacemaker

ii  pacemaker                              1.1.16-1                          amd64        cluster resource manager
ii  pacemaker-cli-utils                    1.1.16-1                          amd64        cluster resource manager command line utilities
ii  pacemaker-common                       1.1.16-1                          all          cluster resource manager common files
ii  pacemaker-resource-agents              1.1.16-1                          all          cluster resource manager general resource agents
ii  pcs                                    0.9.155+dfsg-2+deb9u1             all          Pacemaker Configuration System
ii  resource-agents-paf                    2.3.0-1.pgdg90+1                  all          PostgreSQL resource agent for Pacemaker

oalbrigt commented 4 years ago

grep resource-agents instead to get all the resource-agents packages.

karippery commented 4 years ago

dpkg -l| grep resource-agents

ii  pacemaker-resource-agents              1.1.16-1                          all          cluster resource manager general resource agents
ii  resource-agents                        1:4.0.0~rc1-4                     amd64        Cluster Resource Agents
ii  resource-agents-paf                    2.3.0-1.pgdg90+1                  all          PostgreSQL resource agent for Pacemaker

karippery commented 4 years ago

root@oreo:~# wget https://localhost/server-status
 --2020-09-21 16:47:53--  https://localhost/server-status
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:443... connected.
ERROR: The certificate of 'localhost' is not trusted.
ERROR: The certificate of 'localhost' hasn't got a known issuer.
The certificate's owner does not match hostname 'localhost'
root@oreo:~#

wget --no-check-certificate http://localhost/server-status
--2020-09-21 16:55:49--  http://localhost/server-status
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://10.226.***.***/server-status [following]
--2020-09-21 16:55:49--  https://10.226.***.***/server-status
Connecting to 10.226.***.***:443... connected.
WARNING: The certificate of '10.226.***.***' is not trusted.
WARNING: The certificate of '10.226.***.***' hasn't got a known issuer.
HTTP request sent, awaiting response... 200 OK
Length: 6048 (5.9K) [text/html]
Saving to: 'server-status.1'

server-status.1                         100%[============================================================================>]   5.91K  --.-KB/s    in 0s

2020-09-21 16:55:49 (445 MB/s) - 'server-status.1' saved [6048/6048]

oalbrigt commented 4 years ago

Can you try moving status to localhost:80 or 127.0.0.1:80?

karippery commented 4 years ago

the status location tag in virtualhost :80

<VirtualHost *:80>
            ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html
        Redirect "/" "https://10.226.179.205/"

<Location /server-status>
 SetHandler server-status
 Order deny,allow
 Deny from all
 Allow from 127.0.0.1
</Location>
</VirtualHost>

do you mean this?

oalbrigt commented 4 years ago

The Location section, yeah.

Maybe it'll also work if you put Redirect after </Location>

karippery commented 4 years ago

I changed redirect after location tag

<VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/html <Location /server-status> SetHandler server-status Order deny,allow Deny from all Allow from 127.0.0.1 Redirect "/" "https://10.226.179.205/"

And I do systemctl restart apache2

no differences

 root@oreo:~# pcs status
Cluster name: mycluster
Stack: corosync
Current DC: server3 (version 1.1.16-94ff4df) - partition with quorum
Last updated: Mon Sep 21 17:13:17 2020
Last change: Mon Sep 21 16:22:32 2020 by root via cibadmin on server1

3 nodes configured
5 resources configured

Online: [ server1 server2 server3 ]

Full list of resources:

 MasterVip      (ocf::heartbeat:IPaddr2):       Started server1
 Master/Slave Set: pgsql-ha [pgsqld]
     Masters: [ server1 ]
     Slaves: [ server2 server3 ]
 Apache (ocf::heartbeat:apache):        Stopped

Failed Actions:
* Apache_start_0 on server3 'unknown error' (1): call=315, status=complete, exitreason='Failed to access httpd status page.',
    last-rc-change='Mon Sep 21 16:22:37 2020', queued=0ms, exec=3456ms
* Apache_start_0 on server1 'unknown error' (1): call=59, status=complete, exitreason='Failed to access httpd status page.',
    last-rc-change='Mon Sep 21 16:22:41 2020', queued=0ms, exec=3421ms
* Apache_start_0 on server2 'unknown error' (1): call=197, status=complete, exitreason='Failed to access httpd status page.',
    last-rc-change='Mon Sep 21 16:22:33 2020', queued=0ms, exec=3451ms

oalbrigt commented 4 years ago

Try adding the server-status section in its own VirtualHost section (before *:80 as I think that will catch all domains if you have it before specific domains), example: https://stackoverflow.com/a/19598508

karippery commented 4 years ago

Can I add virtualhost :80 to ssl.conf file (nano /etc/apache2/sites-available/default-ssl.conf )?

<IfModule mod_ssl.c>

<VirtualHost *:80>

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html
        ServerName localhost

        <Location /server-status>
             SetHandler server-status
             Order deny,allow
             Deny from all
             Allow from 127.0.0.1
        </Location>

</VirtualHost>
  <VirtualHost _default_:443>
                ServerAdmin webmaster@localhost

                DocumentRoot /var/www/html

                ErrorLog ${APACHE_LOG_DIR}/error.log
                CustomLog ${APACHE_LOG_DIR}/access.log combined

                SSLEngine on

                SSLCertificateFile      /etc/ssl/certs/apache-selfsigned.crt
                SSLCertificateKeyFile /etc/ssl/private/apache-selfsigned.key

        </VirtualHost>
</IfModule>

karippery commented 4 years ago

or should I create new file like nano /etc/apache2/sites-available/status.conf and add server status section to this file

<VirtualHost *:80>

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html
        ServerName localhost

        <Location /server-status>
             SetHandler server-status
             Order deny,allow
             Deny from all
             Allow from 127.0.0.1
        </Location>

</VirtualHost>

karippery commented 4 years ago

@oalbrigt now I know where is the issue. but i don't know the solution for that. http://localhost/server-status not accessible invalid certificate. maybe this why pacemaker can't access the statusurl.

root@oreo:~# wget --no-check-certificate http://localhost/server-status
--2020-09-22 15:29:53--  http://localhost/server-status
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://10.226.***.***/server-status [following]
--2020-09-22 15:29:53--  https://10.226.179.205/server-status
Connecting to 10.226.***.***:443... connected.
WARNING: The certificate of '10.226.***.***' is not trusted.
WARNING: The certificate of '10.226.***.***' hasn't got a known issuer.
The certificate's owner does not match hostname '10.226.***.***'
HTTP request sent, awaiting response... 403 Forbidden
2020-09-22 15:29:53 ERROR 403: Forbidden.

DO YOU KNOW HOW TO CLEAR THIS ISSUE?

ClusterLabs / resource-agents

How to use Apache resource by HTTPS in pacemaker? #1556