CKAN problems

[VladimirZD]

Hi, we started testing portal with few more users... not too much (10+) but we started getting error (in ckan-apache.error.log) FATAL: sorry, too many clients already, referer:....

We don't have too many clients so i am wondering is there any know issues regarding this?

Any hints how to solve this?

thnx

[davidread]

Can you quote it exactly?

If it is a problem with database connections then I suggest you install pgbouncer to manage them - that's what we do.

https://lists.okfn.org/pipermail/ckan-dev/2013-November/006183.html

But I'm not clear if it is that error.

On 19 March 2015 at 08:40, Vladimir Mašala notifications@github.com wrote:

Hi, we started testing portal with few more users... not too much (10+) but we started getting error (in ckan-apache.error.log) FATAL: sorry, too many clients already, referer:....

We don't have too many clients so i am wondering is there any know issues regarding this?

Any hints how to solve this?

thnx

— Reply to this email directly or view it on GitHub https://github.com/datagovuk/dgu-vagrant-puppet/issues/93.

[tbalaz]

@davidread

Hi,

additional info about this error.

We have made some configuration changes in apache ckan.conf file

#WSGIDaemonProcess singlethreaded #WSGIProcessGroup singlethreaded WSGIProcessGroup %{GLOBAL}

And we have set debug=false in ckan.ini.

After this changes we had to many clients error.

Reverting apache ckan.conf settings back to

WSGIDaemonProcess singlethreaded WSGIProcessGroup singlethreaded #WSGIProcessGroup %{GLOBAL}

resolved error.

Can you tell what is yours configuration in apache ckan.conf file, and how have you configured pgbouncer max_client_conn, default_pool_size?

Also i can't find limit for idle connection in pgbouncer how can i set this?

Thank you.

[davidread]

We tried playing about with WSGIDaemonProcess and WSGIProcessGroup but in the end we just removed these options from our apache config. You may have success with them though.

I've put our pgbouncer config here: https://gist.github.com/davidread/843e355f893a43547e82

And then in the ckan.ini point at pgbouncer:

sqlalchemy.url = postgresql://dgu:REDACTED@localhost:6432/ckan

BTW Please can you quote the exact error message you see in the log, for my benefit and for others.

On 19 March 2015 at 12:53, Tomislav Balaz notifications@github.com wrote:

@davidread https://github.com/davidread

Hi,

additional info about this error.

We have made some configuration changes in apache ckan.conf file

#WSGIDaemonProcess singlethreaded #WSGIProcessGroup singlethreaded WSGIProcessGroup %{GLOBAL}

And we have set debug=false in ckan.ini.

After this changes we had to many clients error.

Reverting apache ckan.conf settings back to

WSGIDaemonProcess singlethreaded WSGIProcessGroup singlethreaded #WSGIProcessGroup %{GLOBAL}

resolved error.

Can you tell what is yours configuration in apache ckan.conf file, and how have you configured pgbouncer max_client_conn, default_pool_size?

Also i can't find limit for idle connection in pgbouncer how can i set this?

Thank you.

— Reply to this email directly or view it on GitHub https://github.com/datagovuk/dgu-vagrant-puppet/issues/93#issuecomment-83544813 .

[VladimirZD]

@davidread David thank you for answer, we had GO LIVE today and this problem happened very near publishing time so it was a bit interesting.. Luckly everything is working fine.... Traceback: " Traceback: (most recent call last):, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/var/ckan/wsgi_app.py", line 9, in , referer: http://data.gov.hr/data/search?sort=metadata_modified+desc application = loadapp('config:%s' % config_filepath), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc return loadobj(APP, uri, name=name, _kw), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc return context.create(), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc return self.object_type.invoke(self), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc return fix_call(context.object, context.global_conf, _context.local_conf), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/paste/deploy/util.py", line 56, in fix_call, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc val = callable(_args, *_kw), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/vagrant/src/ckan/ckan/config/middleware.py", line 61, in make_app, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc app_globals.reset(), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/vagrant/src/ckan/ckan/lib/app_globals.py", line 143, in reset, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc get_config_value(key), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/vagrant/src/ckan/ckan/lib/app_globals.py", line 113, in get_config_value, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc if model.meta.engine.has_table('system_info'):, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2539, in has_table, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc return self.run_callable(self.dialect.has_table, table_name, schema), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2424, in run_callable, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc conn = self.contextual_connect(), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2490, in contextual_connect, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc self.pool.connect(),, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/sqlalchemy/pool.py", line 224, in connect, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc return _ConnectionFairy(self).checkout(), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/sqlalchemy/pool.py", line 387, in init, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc rec = self._connection_record = pool._do_get(), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/sqlalchemy/pool.py", line 741, in _do_get, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc con = self._create_connection(), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/sqlalchemy/pool.py", line 188, in _create_connection, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc return _ConnectionRecord(self), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/sqlalchemy/pool.py", line 270, in init, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc self.connection = self.connect(), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/sqlalchemy/pool.py", line 330, in __connect, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc connection = self.pool._creator(), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 80, in connect, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc return dialect.connect(_cargs, _cparams), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 281, in connect, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc return self.dbapi.connect(_cargs, _cparams), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc File "/home/co/ckan/lib/python2.7/site-packages/psycopg2/init.py", line 179, in connect, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc connection_factory=connection_factory, async=async), referer: http://data.gov.hr/data/search?sort=metadata_modified+desc OperationalError: (OperationalError) FATAL: sorry, too many clients already, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc FATAL: sorry, too many clients already, referer: http://data.gov.hr/data/search?sort=metadata_modified+desc "

Thank you for help