search

vmware-archive / sre-test

Greenplum - Open Source SRE test project.
2 stars 0 forks source link

log_rotation_size parameter issue #217

Closed bala-cg closed 2 years ago

bala-cg commented 3 years ago

Page No : 1366

Issue: Post setting this parameter log_rotation_size on restart, DB did not start, DB started after reverting this change.

[gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ gpconfig -c log_rotation_size -v 2GB 20210725:16:17:45:015953 gpconfig:mdw_ipv4:gpadmin-[INFO]:-completed successfully with parameters '-c log_rotation_size -v 2GB' [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ gpconfig -s log_rotation_size Values on all segments are consistent GUC : log_rotation_size Master value: 1GB Segment value: 1GB [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ gpstop -u 20210725:16:21:32:016296 gpstop:mdw_ipv4:gpadmin-[INFO]:-Starting gpstop with args: -u 20210725:16:21:32:016296 gpstop:mdw_ipv4:gpadmin-[INFO]:-Gathering information and validating the environment... 20210725:16:21:32:016296 gpstop:mdw_ipv4:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information 20210725:16:21:32:016296 gpstop:mdw_ipv4:gpadmin-[INFO]:-Obtaining Segment details from master... 20210725:16:21:32:016296 gpstop:mdw_ipv4:gpadmin-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 6.16.0 build commit:5650be2b79197fed564dca8d734d10f2a76b876c' 20210725:16:21:32:016296 gpstop:mdw_ipv4:gpadmin-[INFO]:-Signalling all postmaster processes to reload [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ gpconfig -s log_rotation_size Values on all segments are consistent GUC : log_rotation_size Master value: 1GB Segment value: 1GB [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ gpstop -arf 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Starting gpstop with args: -arf 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Gathering information and validating the environment... 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Obtaining Segment details from master... 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 6.16.0 build commit:5650be2b79197fed564dca8d734d10f2a76b876c' 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Commencing Master instance shutdown with mode='fast' 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Master segment instance directory=/data1/gpdb/master/gpseg-1 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Attempting forceful termination of any leftover master process 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Terminating processes for segment /data1/gpdb/master/gpseg-1 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-No standby master host configured 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Targeting dbid [2, 8, 3, 9, 4, 10, 5, 11, 6, 12, 7, 13] for shutdown 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Commencing parallel primary segment instance shutdown, please wait... 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-0.00% of jobs completed 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-100.00% of jobs completed 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Commencing parallel mirror segment instance shutdown, please wait... 20210725:16:21:45:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-0.00% of jobs completed 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-100.00% of jobs completed 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:----------------------------------------------------- 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:- Segments stopped successfully = 12 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:- Segments with errors during stop = 0 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:----------------------------------------------------- 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Successfully shutdown 12 of 12 segment instances 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Database successfully shutdown with no errors reported 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Cleaning up leftover gpmmon process 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-No leftover gpmmon process found 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Cleaning up leftover gpsmon processes 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-No leftover gpsmon processes on some hosts. not attempting forceful termination on these hosts 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Cleaning up leftover shared memory 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[INFO]:-Restarting System... 20210725:16:21:46:016346 gpstop:mdw_ipv4:gpadmin-[CRITICAL]:-Error occurred: non-zero rc: 2 Command was: '$GPHOME/bin/gpstart -a -d /data1/gpdb/master/gpseg-1' rc=2, stdout='20210725:16:21:46:016479 gpstart:mdw_ipv4:gpadmin-[INFO]:-Starting gpstart with args: -a -d /data1/gpdb/master/gpseg-1 20210725:16:21:46:016479 gpstart:mdw_ipv4:gpadmin-[INFO]:-Gathering information and validating the environment... 20210725:16:21:46:016479 gpstart:mdw_ipv4:gpadmin-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 6.16.0 build commit:5650be2b79197fed564dca8d734d10f2a76b876c' 20210725:16:21:46:016479 gpstart:mdw_ipv4:gpadmin-[INFO]:-Greenplum Catalog Version: '301908232' 20210725:16:21:46:016479 gpstart:mdw_ipv4:gpadmin-[INFO]:-Starting Master instance in admin mode 20210725:16:21:46:016479 gpstart:mdw_ipv4:gpadmin-[CRITICAL]:-Failed to start Master instance in admin mode 20210725:16:21:46:016479 gpstart:mdw_ipv4:gpadmin-[CRITICAL]:-Error occurred: non-zero rc: 1 Command was: 'env GPSESSID=0000000000 GPERA=None $GPHOME/bin/pg_ctl -D /data1/gpdb/master/gpseg-1 -l /data1/gpdb/master/gpseg-1/pg_log/startup.log -w -t 600 -o " -p 5432 -c gp_role=utility " start' rc=1, stdout='waiting for server to start.... stopped waiting ', stderr='pg_ctl: could not start server Examine the log output. ' ', stderr='' [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$

[gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$ grep log_rotation_size $MASTER_DATA_DIRECTORY/postgresql.conf

log_rotation_size = 10MB # Automatic rotation of logfiles will

log_rotation_size=2GB

[gpadmin@mdw_ipv4 pg_log]$ [gpadmin@mdw_ipv4 pg_log]$

fifthposition commented 3 years ago

@bala-cg Is this a product issue / bug?

bala-cg commented 3 years ago

@fifthposition, yes this is a bug, updated the label.