search

poeml / mirrorbrain

MirrorBrain
http://mirrorbrain.org/
Other
75 stars 37 forks source link

mod_stats causing mirrorbrain dbd error? #178

Open chrisvwn opened 5 years ago

chrisvwn commented 5 years ago

I am aware that mod_stats has not been maintained for a while now, however, I am looking at possibly modifying it to log realtime stats for an mb installation.

mod_stats builds fine and seems to run, however, I have noticed that when I enable it the mod_dbd connections fail for both mb and mod_stats. Is there a reason that this should happen? Maybe there is something outdated?

Apache is using mpm_event and mod_dbd.

This is what the mb logs look like without mod_stats:

[Thu Feb 21 23:20:32.403434 2019] [mirrorbrain:warn] [pid 26242:tid 140033674495744] [client client.mbdomain.com:40110] [mod_mirrorbrain] MirrorBrainEngine On, mirror_base '/srv/mbroot/'
[Thu Feb 21 23:20:32.403497 2019] [mirrorbrain:warn] [pid 26242:tid 140033674495744] [client client.mbdomain.com:40110] [mod_mirrorbrain] URI: '/'
[Thu Feb 21 23:20:32.403505 2019] [mirrorbrain:warn] [pid 26242:tid 140033674495744] [client client.mbdomain.com:40110] [mod_mirrorbrain] filename: '/srv/cdnzoo/'
[Thu Feb 21 23:20:32.403555 2019] [mirrorbrain:warn] [pid 26242:tid 140033674495744] [client client.mbdomain.com:40110] [mod_mirrorbrain] clientip: client.mbdomain.com
[Thu Feb 21 23:20:32.403565 2019] [mirrorbrain:warn] [pid 26242:tid 140033674495744] [client client.mbdomain.com:40110] [mod_mirrorbrain] '/srv/mbroot/' is a directory
[Thu Feb 21 23:20:38.961703 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] MirrorBrainEngine On, mirror_base '/srv/mbroot/', referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:38.961727 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] URI: '/CEP389.mpg', referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:38.961735 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] filename: '/srv/cdnzoo/CEP389.mpg', referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:38.961781 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] clientip: client.mbdomain.com, referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:38.961801 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] Country 'KE', Continent 'AF', referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:38.961812 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] AS '--', Prefix '--', lat/lng 1.000000,38.000000 state id (null), state '(null)', referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:38.961833 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] Canonicalized file on disk: /srv/mbroot/CEP389.mpg, referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:38.961843 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] SQL file to look up: CEP389.mpg, referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:38.975988 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] Successfully acquired database connection., referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:39.137658 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] Found 1 mirror, referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:39.137693 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] [mod_mirrorbrain] taking geo distance into account, referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:39.137703 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] elsewhere:    mirror.mbdomain.com              (score  100) (rank     846930) (dist 60982), referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:39.137711 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] classifying 1 mirror: 0 prefix, 0 AS, 0 country, 0 region, 1 elsewhere, referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:39.165834 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] Chose server mirror.mbdomain.com, referer: http://mbserver.mbdomain.com/
[Thu Feb 21 23:20:39.165853 2019] [mirrorbrain:warn] [pid 26242:tid 140033583478528] [client client.mbdomain.com:40116] [mod_mirrorbrain] Redirect to 'http://mirror.mbdomain.com/CEP389.mpg', referer: http://mbserver.mbdomain.com/

However, once I enable mod_stats, both mb and mod_stats log errors related to dbd. For convenience the error line is:

Internal error (specific information not available): AH02655: Failed to acquire DBD connection from pool!

The full logs

[Fri Feb 22 09:42:58.581983 2019] [mirrorbrain:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_mirrorbrain] MirrorBrainEngine On, mirror_base '/srv/mbroot/', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:42:58.582007 2019] [mirrorbrain:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_mirrorbrain] URI: '/CEP389.mpg', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:42:58.582015 2019] [mirrorbrain:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_mirrorbrain] filename: '/srv/mbroot/CEP389.mpg', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:42:58.582062 2019] [mirrorbrain:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_mirrorbrain] clientip: client.mbdomain.com, referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:42:58.582083 2019] [mirrorbrain:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_mirrorbrain] Country 'MU', Continent 'AF', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:42:58.582095 2019] [mirrorbrain:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_mirrorbrain] AS '--', Prefix '--', lat/lng -20.283300,57.549999 state id (null), state '(null)', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:42:58.582120 2019] [mirrorbrain:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_mirrorbrain] Canonicalized file on disk: /srv/mbroot/CEP389.mpg, referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:42:58.582136 2019] [mirrorbrain:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_mirrorbrain] SQL file to look up: CEP389.mpg, referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:42:58.597901 2019] [dbd:error] [pid 8752:tid 140539806099200] (20014)Internal error (specific information not available): AH02655: Failed to acquire DBD connection from pool!
[Fri Feb 22 09:43:05.838251 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] Stats enabled, stats_base '/srv/mbroot/', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838279 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] filename: '/srv/mbroot/CEP389.mpg', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838286 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] uri: '/CEP389.mpg', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838293 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] req_filename: 'CEP389.mpg', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838314 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] stats_parse_req(): file: 'CEP389.mpg', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838330 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] stats_parse_req(): file: 'CEP389' after stripping type, referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838336 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] stats_parse_req(): file: 'CEP389' after stripping arch, referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838344 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] stats_parse_req(): file: 'CEP389' after stripping release, referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838350 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] stats_parse_req(): file: 'CEP389' after stripping version, referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838356 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] fname:   'CEP389.mpg', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838362 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] project: 'CEP389.mpg', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838368 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] repo:    'CEP389.mpg', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838374 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] package: 'CEP389', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838379 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] version: 'CEP389', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838385 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] release: 'CEP389', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838391 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] arch:    'CEP389.mpg', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.838397 2019] [stats:warn] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] type:    'mpg', referer: http://mbserver.mbdomain.com/
[Fri Feb 22 09:43:05.851495 2019] [dbd:error] [pid 8752:tid 140539806099200] (20014)Internal error (specific information not available): AH02655: Failed to acquire DBD connection from pool!
[Fri Feb 22 09:43:05.851523 2019] [stats:error] [pid 8752:tid 140539806099200] [client client.mbdomain.com:40680] [mod_stats] Error acquiring database connection, referer: http://mbserver.mbdomain.com/

Any assistance would be appreciated.

poeml commented 5 years ago

mod_stats was probably used with MySQL, before the switch to PostgreSQL. Also, it was maybe used with an earlier (very early) mod_dbd API. Apache DBD still was young in the days.

I suggest to check stats_dbd_prepare() carefully, if it uses the correct semantics. You could compare to mod_mirrorbrain (where it works).

Another question: did you configure queries as can be seen in https://github.com/poeml/mirrorbrain/blob/master/mod_stats/mod_stats.conf ?

I don’t quite understand how our modules trigger this seemingly dbd-internal error.

Am 22.02.2019 um 15:24 schrieb Chris Njuguna notifications@github.com:

I am aware that mod_stats has not been maintained for a while now, however, I am looking at possibly modifying it to log realtime stats for an mb installation.

mod_stats builds fine and seems to run, however, I have noticed that when I enable it the mod_dbd connections fail for both mb and mod_stats. Is there a reason that this should happen? Maybe there is something outdated?

Apache is using mpm_event and mod_dbd.

This is what the mb logs look like without mod_stats:

chrisvwn commented 5 years ago

Thanks @poeml I will look at the stats_dbd_prepare().

I did configure the queries - rewritten for postgresql. These are the queries I have - are they alright?

    StatsDBDQuery "UPDATE redirect_stats SET count=count+1, counted_at=NOW() WHERE project=%s AND repository=%s AND arch=%s AND filename=%s AND filetype=%s AND version=%s AND release=%s"

    StatsDBDSelectQuery "SELECT id FROM redirect_stats WHERE project=%s AND repository=%s AND arch=%s AND filename=%s AND filetype=%s AND version=%s AND release=%s"

    StatsDBDDeleteQuery "DELETE FROM redirect_stats WHERE project=%s AND repository=%s AND arch=%s AND filename=%s AND filetype=%s AND version=%s AND release=%s"

    StatsDBDInsertQuery "INSERT INTO redirect_stats (id, project, repository, arch, filename, filetype, version, release, counted_at, package) VALUES (%i, %s, %s, %s, %s, %s, %s, %s, NOW(), %s)"

I am perplexed how enabling mod_stats affects mb via dbd.

chrisvwn commented 5 years ago

I have just realized that I had installed mirrorbrain from the OpenSUSE mirrors at http://download.opensuse.org/repositories/Apache:/MirrorBrain/xUbuntu_15.10/ Meanwhile, I downloaded mod_stats from here on github since the packages did not have mod_stats. I have just realized what I have installed is mb version 2.18.1 while the master seems to be at 2.19.0. Is this likely to cause a problem? Do I have to upgrade the mirrorbrain?