SQLite logging error by concurrent users

[geoslegend]

Description

I set my httpd config file using wsgi setting values on pywps readme. When concurrent users increase, the error message is shown as follows:

[Mon Aug 08 14:43:30.760881 2016] [wsgi:error] [pid 10745:tid 140681473595136] [remote xxx.xxx.xxx.xxx:17141] mod_wsgi (pid=10745): Exception occurred processing WSGI script '/xxx/pywps/wps.wsgi'.
[Mon Aug 08 14:43:30.761160 2016] [wsgi:error] [pid 10745:tid 140681473595136] [remote xxx.xxx.xxx.xxx:17141] Traceback (most recent call last):
[Mon Aug 08 14:43:30.761509 2016] [wsgi:error] [pid 10745:tid 140681473595136] [remote xxx.xxx.xxx.xxx:17141]   File "/xxx/python2.7/site-packages/werkzeug/wrappers.py", line 291, in application
[Mon Aug 08 14:43:30.765715 2016] [wsgi:error] [pid 10745:tid 140681473595136] [remote xxx.xxx.xxx.xxx:17141]     return f(*args[:-2] + (request,))(*args[-2:])
[Mon Aug 08 14:43:30.765741 2016] [wsgi:error] [pid 10745:tid 140681473595136] [remote xxx.xxx.xxx.xxx:17141]   File "/xxx/python2.7/site-packages/pywps-4.0.0a2-py2.7.egg/pywps/app/Service.py", line 573, in __call__
[Mon Aug 08 14:43:30.765962 2016] [wsgi:error] [pid 10745:tid 140681473595136] [remote xxx.xxx.xxx.xxx:17141]     log_request(request_uuid, wps_request)
[Mon Aug 08 14:43:30.765976 2016] [wsgi:error] [pid 10745:tid 140681473595136] [remote xxx.xxx.xxx.xxx:17141]   File "/xxx/python2.7/site-packages/pywps-4.0.0a2-py2.7.egg/pywps/dblog.py", line 38, in log_request
[Mon Aug 08 14:43:30.766059 2016] [wsgi:error] [pid 10745:tid 140681473595136] [remote xxx.xxx.xxx.xxx:17141]     cur = conn.cursor()
[Mon Aug 08 14:43:30.766081 2016] [wsgi:error] [pid 10745:tid 140681473595136] [remote xxx.xxx.xxx.xxx:17141] ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140681515554560 and this is thread

Environment

• operating system: CentOS 6.5
• Python version: 2.7.10
• PyWPS version: 4.0.0beta1
• source/distribution
  • [ O] git clone
  • [ ] Debian
  • [ ] PyPI
  • [ ] zip/tar.gz
  • [ ] other (please specify):
• web server
  • [ O] Apache/mod_wsgi
  • [ ] CGI
  • [ ] other (please specify):

    Steps to Reproduce

    Additional Information

[jachym]

Hi @geoslegend, thanks for the report. I'm afraid, I can not reproduce this bug - could you please share the process (if possible), you are testing with and the request, you are trying to pass?

[geoslegend]

@jachym I found the similar case: http://stackoverflow.com/questions/393554/python-sqlite3-and-concurrency Please check that.

[ldesousa]

Please try this solution: http://stackoverflow.com/a/2894830/2066215

[jachym]

Looking into the code,

pywps/dblog.py:        _CONNECTION = sqlite3.connect(database, check_same_thread=False)

seems to be implemented ...

shell we just adopt work of @jan-rudolf , who introduced sqlalchemy along with support for postgresql?

https://github.com/jan-rudolf/pywps/blob/master/pywps/dblog.py

But this does not seem to be 1:1 translation

[ldesousa]

Here is a discussion on SQLite providing a few interesting alternatives: http://stackoverflow.com/q/417917/2066215

I woked with Firebird a long time ago and it was quite solid. There is a Python library for Firebird: https://pyfirebirdsql.readthedocs.io/en/latest/index.html

[ldesousa]

Another alternative might be Redis: https://pypi.python.org/pypi/redis/