ERROR [django.request:124] Internal Server Error:/catalog/submit

[tpegues2]

After upgrading from SAL 3.3.5 to SAL 3.3.8, we've noticed the following errors in our sal.log file and some slowness as well as time outs have started to occur. Any recommendations?

[10/Aug/2018 14:53:33] ERROR [django.request:124] Internal Server Error: /catalog/submit/ Traceback (most recent call last): File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, callback_kwargs) File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view return view_func(*args, *kwargs) File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/views/decorators/http.py", line 40, in inner return func(request, args, kwargs) File "/srv/saluser/sal_env/sal/sal/decorators.py", line 170, in wrap return function(request, *args, *kwargs) File "/srv/saluser/sal_env/sal/catalog/views.py", line 29, in submit_catalog submission = request.POST File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 128, in _get_post self._load_post_and_files() File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/http/request.py", line 310, in _load_post_and_files self._post, self._files = QueryDict(self.body, encoding=self._encoding), MultiValueDict() File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/http/request.py", line 273, in body six.reraise(UnreadablePostError, UnreadablePostError(e.args), sys.exc_info()[2]) File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/http/request.py", line 271, in body self._body = self.read() File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/http/request.py", line 332, in read six.reraise(UnreadablePostError, UnreadablePostError(e.args), sys.exc_info()[2]) File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/http/request.py", line 330, in read return self._stream.read(args, **kwargs) File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 49, in read result = self.buffer + self._read_limited() File "/srv/saluser/sal_env/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 43, in _read_limited result = self.stream.read(size) File "/srv/saluser/sal_env/lib/python2.7/site-packages/gunicorn/http/body.py", line 212, in read data = self.reader.read(1024) File "/srv/saluser/sal_env/lib/python2.7/site-packages/gunicorn/http/body.py", line 123, in read data = self.unreader.read() File "/srv/saluser/sal_env/lib/python2.7/site-packages/gunicorn/http/unreader.py", line 38, in read d = self.chunk() File "/srv/saluser/sal_env/lib/python2.7/site-packages/gunicorn/http/unreader.py", line 65, in chunk return self.sock.recv(self.mxchunk) UnreadablePostError: [Errno 104] Connection reset by peer

Also in the apache logs we see these all day:

[Sun Aug 12 06:01:23.830594 2018] [proxy:error] [pid 11223] AH00940: HTTP: disabled connection for (localhost) [Sun Aug 12 06:01:23.836827 2018] [proxy:error] [pid 11234] AH00940: HTTP: disabled connection for (localhost) [Sun Aug 12 06:01:23.837907 2018] [proxy:error] [pid 11236] AH00940: HTTP: disabled connection for (localhost) [Sun Aug 12 06:01:23.841641 2018] [proxy:error] [pid 11232] AH00940: HTTP: disabled connection for (localhost) [Sun Aug 12 06:01:23.850094 2018] [proxy:error] [pid 11233] AH00940: HTTP: disabled connection for (localhost) [Sun Aug 12 06:01:23.858806 2018] [proxy:error] [pid 11238] AH00940: HTTP: disabled connection for (localhost) [Sun Aug 12 06:01:23.874125 2018] [proxy:error] [pid 11085] (70007)The timeout specified has expired: AH00957: HTTP: attempt to connect to 127.0.0.1:8000 (localhost) failed [Sun Aug 12 06:01:23.874177 2018] [proxy:error] [pid 11085] AH00959: ap_proxy_connect_backend disabling worker for (localhost) for 60s

[grahamgilbert]

This is odd as the docker container doesn’t use Apache? Assuming you are using the legacy install method, you should be looking at the specs for both the app server and more importantly the database. If there is an error with submitting catalogs, you should post up a copy of the offending catalog so we can see what might be different about it.

[tpegues2]

How do we submit a catalog? I'm not exactly sure how we determine this. Here is the guide that we used: https://github.com/salopensource/sal/wiki/RHEL-7

[grahamgilbert]

The catalog is the munki catalog. I don't use RHEL, so unless the app is giving specific errors it's hard to debug. Do you see this when using the Docker image? If I had to guess, I would say you have a large catalog and the proxy is timing out before the app gets the whole file.

[grahamgilbert]

Bump @tpegues2

[tpegues2]

We are not using the Docker image. We were able to bump up the memory that our PostgresSQL uses and the slowness has disappeared. It is possible the clients need to be upgraded and with doing so these may go away. Thanks for your help. :)