search

dmwm / DMWMMON

1 stars 8 forks source link

Investigate about server side errors for dmwmmon data service #25

Open nataliaratnikova opened 8 years ago

nataliaratnikova commented 8 years ago

Look at the errors logged by cmsweb server, classify and investigate.

nataliaratnikova commented 8 years ago

As of last week. most frequent error is about size limit, e.g.:

Fri Jan 29 00:45:27 2016 Apache2::SizeLimit httpd process too big, exiting at SIZE=409964 KB SHARE=3292 KB UNSHARED=406672 REQUESTS=495 LIFETIME=85879 seconds

Other errors are these:

-sh-4.1$ cat _/dmwmmon/error_log__ | sort -u | grep -v Apache2::Size [Fri Jan 29 13:59:14 2016] [notice] Apache/2.2.31 (Unix) mod_ssl/2.2.31 OpenSSL/1.0.1p mod_perl/2.0.9 Perl/v5.10.1 configured -- resuming normal operations [Fri Jan 29 13:59:40 2016] [notice] Apache/2.2.31 (Unix) mod_ssl/2.2.31 OpenSSL/1.0.1p mod_perl/2.0.9 Perl/v5.10.1 configured -- resuming normal operations Issuing rollback() due to DESTROY without explicit disconnect() of DBD::Oracle::db handle cms_sitedb at /data/srv/beHG1602d-comp/sw.coimbra/slc6_amd64_gcc493/external/p5-apache-dbi/1.08/lib/perl5/Apache/DBI.pm line 189. Issuing rollback() due to DESTROY without explicit disconnect() of DBD::Oracle::db handle cms_sitedb at /data/srv/beHG1602f/sw/slc6_amd64_gcc493/external/p5-apache-dbi/1.08/lib/perl5/Apache/DBI.pm line 189. [Thu Jan 28 18:18:47 2016] [notice] Apache/2.2.31 (Unix) mod_ssl/2.2.31 OpenSSL/1.0.1p mod_perl/2.0.9 Perl/v5.10.1 configured -- resuming normal operations [Thu Jan 28 18:18:48 2016] [notice] Apache/2.2.31 (Unix) mod_ssl/2.2.31 OpenSSL/1.0.1p mod_perl/2.0.9 Perl/v5.10.1 configured -- resuming normal operations [Tue Feb 02 12:12:03 2016] [notice] Apache/2.2.31 (Unix) mod_ssl/2.2.31 OpenSSL/1.0.1p mod_perl/2.0.9 Perl/v5.10.1 configured -- resuming normal operations [Tue Feb 02 14:58:42 2016] [notice] Apache/2.2.31 (Unix) mod_ssl/2.2.31 OpenSSL/1.0.1p mod_perl/2.0.9 Perl/v5.10.1 configured -- resuming normal operations [Tue Feb 02 14:58:49 2016] [notice] Apache/2.2.31 (Unix) mod_ssl/2.2.31 OpenSSL/1.0.1p mod_perl/2.0.9 Perl/v5.10.1 configured -- resuming normal operations [Tue Feb 02 14:58:59 2016] [notice] Apache/2.2.31 (Unix) mod_ssl/2.2.31 OpenSSL/1.0.1p mod_perl/2.0.9 Perl/v5.10.1 configured -- resuming normal operations Use of uninitialized value in subroutine entry at /data/srv/beHG1602d-comp/sw.coimbra/slc6_amd64_gcc493/cms/DMWMMON-datasvc/1.0.5/perl_lib/PHEDEX/Web/FrontendAuth.pm line 39. Use of uninitialized value in subroutine entry at /data/srv/beHG1602e/sw/slc6_amd64_gcc493/cms/DMWMMON-datasvc/1.1.0/perl_lib/PHEDEX/Web/FrontendAuth.pm line 39. Use of uninitialized value in subroutine entry at /data/srv/beHG1602f/sw/slc6_amd64_gcc493/cms/DMWMMON-datasvc/1.1.0/perl_lib/PHEDEX/Web/FrontendAuth.pm line 39. Using default DB access for 'dumpspacequery' API Using default DB access for 'getlastrecord' API Using default DB access for 'nodes' API -sh-4.1$

2

Updated by Natalia Ratnikova 2 days ago Comment Edit

This error:

Use of uninitialized value in subroutine entry at /data/srv/beHG1602f/sw/slc6_amd64_gcc493/cms/DMWMMON-datasvc/1.1.0/perl_lib/PHEDEX/Web/FrontendAuth.pm line 39

happens regularly on two servers:

-sh-4.1$ grep -c "FrontendAuth.pm line 39" vocms/dmwmmon/error vocms0131/dmwmmon/error_log_20160127.txt:8 vocms0131/dmwmmon/error_log_20160128.txt:10 vocms0131/dmwmmon/error_log_20160129.txt:4 vocms0131/dmwmmon/error_log_20160130.txt:2 vocms0131/dmwmmon/error_log_20160131.txt:10 vocms0131/dmwmmon/error_log_20160201.txt:10 vocms0131/dmwmmon/error_log_20160202.txt:6 vocms0132/dmwmmon/error_log_20160127.txt:2 vocms0132/dmwmmon/error_log_20160128.txt:10 vocms0132/dmwmmon/error_log_20160129.txt:12 vocms0132/dmwmmon/error_log_20160130.txt:6 vocms0132/dmwmmon/error_log_20160131.txt:2 vocms0132/dmwmmon/error_log_20160201.txt:6 vocms0132/dmwmmon/error_log_20160202.txt:8 vocms0136/dmwmmon/error_log_20160127.txt:0 vocms0136/dmwmmon/error_log_20160129.txt:0 vocms0136/dmwmmon/error_log_20160130.txt:0 vocms0136/dmwmmon/error_log_20160202.txt:0 vocms0161/dmwmmon/error_log_20160127.txt:0 vocms0161/dmwmmon/error_log_20160128.txt:0 vocms0161/dmwmmon/error_log_20160129.txt:0 vocms0161/dmwmmon/error_log_20160131.txt:0 vocms0161/dmwmmon/error_log_20160202.txt:0 vocms0163/dmwmmon/error_log_20160127.txt:0 vocms0163/dmwmmon/error_log_20160128.txt:0 vocms0163/dmwmmon/error_log_20160129.txt:0 vocms0163/dmwmmon/error_log_20160130.txt:0 vocms0163/dmwmmon/error_log_20160201.txt:0 vocms0163/dmwmmon/error_log_20160202.txt:0 vocms0165/dmwmmon/error_log_20160126.txt:0 vocms0165/dmwmmon/error_log_20160127.txt:0 vocms0165/dmwmmon/error_log_20160128.txt:0 vocms0165/dmwmmon/error_log_20160130.txt:0 vocms0165/dmwmmon/error_log_20160201.txt:0 vocms0165/dmwmmon/error_log_20160202.txt:0

The corresponding code line in perl_lib/PHEDEX/Web/FrontendAuth.pm is:

$self->{HEADER}{lc $_} = unacstring('utf-8',$headers->{$});