squeaky-pl / japronto

Screaming-fast Python 3.5+ HTTP toolkit integrated with pipelining HTTP server based on uvloop and picohttpparser.
MIT License
8.61k stars 581 forks source link

Memory leak? #112

Open carniz opened 6 years ago

carniz commented 6 years ago

I'm trying out japronto 0.1.1, and I have a small app looking like this:

# -*- coding: utf-8 -*-

import os
import requests
import japronto
import multiprocessing

WIREMOCK_URL = os.environ.get("WIREMOCK_URL", "http://wiremock:8080")

def index(request):
    return request.Response(text='Welcome to the Japronto MicroService')

def pong(request):
    return request.Response(text='pong')

def wiremock(request):
    response = requests.get("%s/get/this" % WIREMOCK_URL).text
    return request.Response(text=response)

app = japronto.Application()
app.router.add_route('/', index)
app.router.add_route('/ping', pong)
app.router.add_route('/wiremock', wiremock)
app.run(worker_num=multiprocessing.cpu_count()*2 + 1)

It is docker-composed together with a wiremock-container, which is stubbed like this:

curl -X POST --data '{ "request": { "url": "/get/this", "method": "GET" }, "response": { "status": 200, "body": "Ecco ci\n" }}' localhost:8080/__admin/mappings/new

I then run apache-bench like this:

ab -n 10000 -c100 localhost:9090/wiremock

The first run, it completes in 9.2 seconds:


$ ab -n 10000 -c100 localhost:9090/wiremock
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests

Server Software:        
Server Hostname:        localhost
Server Port:            9090

Document Path:          /wiremock
Document Length:        8 bytes

Concurrency Level:      100
Time taken for tests:   9.207 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      870000 bytes
HTML transferred:       80000 bytes
Requests per second:    1086.15 [#/sec] (mean)
Time per request:       92.068 [ms] (mean)
Time per request:       0.921 [ms] (mean, across all concurrent requests)
Transfer rate:          92.28 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.7      0       7
Processing:     5   91  81.9     72     711
Waiting:        5   90  81.5     71     710
Total:          5   91  82.1     73     714
WARNING: The median and mean for the initial connection time are not within a normal deviation
        These results are probably not that reliable.

Percentage of the requests served within a certain time (ms)
  50%     73
  66%    101
  75%    120
  80%    136
  90%    180
  95%    254
  98%    295
  99%    361
 100%    714 (longest request)

The second run, it completes in 13.5 seconds:

$ ab -n 10000 -c100 localhost:9090/wiremock
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests

Server Software:        
Server Hostname:        localhost
Server Port:            9090

Document Path:          /wiremock
Document Length:        8 bytes

Concurrency Level:      100
Time taken for tests:   13.500 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      870000 bytes
HTML transferred:       80000 bytes
Requests per second:    740.76 [#/sec] (mean)
Time per request:       134.996 [ms] (mean)
Time per request:       1.350 [ms] (mean, across all concurrent requests)
Transfer rate:          62.94 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.6      0       6
Processing:     6  134 109.2    104     662
Waiting:        6  133 108.7    104     659
Total:          6  134 109.4    105     662

Percentage of the requests served within a certain time (ms)
  50%    105
  66%    150
  75%    186
  80%    211
  90%    290
  95%    366
  98%    428
  99%    469
 100%    662 (longest request)

The third time, it doesn't complete:

$ ab -n 10000 -c100 localhost:9090/wiremock
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
apr_socket_recv: Connection reset by peer (104)
Total of 8631 requests completed

The docker-compose logs reveals the following:

$ docker-compose logs
Attaching to sonic_suthost_1, sonic_wiremock_1
suthost_1   | Accepting connections on http://0.0.0.0:8080
suthost_1   | Accepting connections on http://0.0.0.0:8080
suthost_1   | Accepting connections on http://0.0.0.0:8080
suthost_1   | Accepting connections on http://0.0.0.0:8080
suthost_1   | Accepting connections on http://0.0.0.0:8080
suthost_1   | Accepting connections on http://0.0.0.0:8080
suthost_1   | Accepting connections on http://0.0.0.0:8080
suthost_1   | Accepting connections on http://0.0.0.0:8080
suthost_1   | Accepting connections on http://0.0.0.0:8080
wiremock_1  | 2018-02-25 11:00:51.910:INFO::Logging to STDERR via wiremock.org.mortbay.log.StdErrLog
suthost_1   | Accepting connections on http://0.0.0.0:8080
wiremock_1  |  /$$      /$$ /$$                     /$$      /$$                     /$$      
suthost_1   | Accepting connections on http://0.0.0.0:8080
wiremock_1  | | $$  /$ | $$|__/                    | $$$    /$$$                    | $$      
suthost_1   | Accepting connections on http://0.0.0.0:8080
wiremock_1  | | $$ /$$$| $$ /$$  /$$$$$$   /$$$$$$ | $$$$  /$$$$  /$$$$$$   /$$$$$$$| $$   /$$
suthost_1   | Accepting connections on http://0.0.0.0:8080
wiremock_1  | | $$/$$ $$ $$| $$ /$$__  $$ /$$__  $$| $$ $$/$$ $$ /$$__  $$ /$$_____/| $$  /$$/
suthost_1   | Accepting connections on http://0.0.0.0:8080
wiremock_1  | | $$$$_  $$$$| $$| $$  \__/| $$$$$$$$| $$  $$$| $$| $$  \ $$| $$      | $$$$$$/ 
suthost_1   | Accepting connections on http://0.0.0.0:8080
wiremock_1  | | $$$/ \  $$$| $$| $$      | $$_____/| $$\  $ | $$| $$  | $$| $$      | $$_  $$ 
suthost_1   | Accepting connections on http://0.0.0.0:8080
wiremock_1  | | $$/   \  $$| $$| $$      |  $$$$$$$| $$ \/  | $$|  $$$$$$/|  $$$$$$$| $$ \  $$
suthost_1   | Accepting connections on http://0.0.0.0:8080
wiremock_1  | |__/     \__/|__/|__/       \_______/|__/     |__/ \______/  \_______/|__/  \__/
suthost_1   | Traceback (most recent call last):
wiremock_1  | 
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
wiremock_1  | port:                         8080
wiremock_1  | enable-browser-proxying:      false
suthost_1   |     (self.host, self.port), self.timeout, **extra_kw)
wiremock_1  | no-request-journal:           false
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
wiremock_1  | verbose:                      false
suthost_1   |     raise err
wiremock_1  | 
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
suthost_1   |     sock.connect(sa)
suthost_1   | OSError: [Errno 99] Cannot assign requested address
suthost_1   | 
suthost_1   | During handling of the above exception, another exception occurred:
suthost_1   | 
suthost_1   | Traceback (most recent call last):
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
suthost_1   |     chunked=chunked)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
suthost_1   |     conn.request(method, url, **httplib_request_kw)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
suthost_1   |     self._send_request(method, url, body, headers, encode_chunked)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
suthost_1   |     self.endheaders(body, encode_chunked=encode_chunked)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
suthost_1   |     self._send_output(message_body, encode_chunked=encode_chunked)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 1026, in _send_output
suthost_1   |     self.send(msg)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 964, in send
suthost_1   |     self.connect()
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
suthost_1   |     conn = self._new_conn()
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
suthost_1   |     self, "Failed to establish a new connection: %s" % e)
suthost_1   | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff3c7bcbfd0>: Failed to establish a new connection: [Errno 99] Cannot assign requested address
suthost_1   | 
suthost_1   | During handling of the above exception, another exception occurred:
suthost_1   | 
suthost_1   | Traceback (most recent call last):
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
suthost_1   |     timeout=timeout
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
suthost_1   |     _stacktrace=sys.exc_info()[2])
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
suthost_1   |     raise MaxRetryError(_pool, url, error or ResponseError(cause))
suthost_1   | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='wiremock', port=8080): Max retries exceeded with url: /get/this (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff3c7bcbfd0>: Failed to establish a new connection: [Errno 99] Cannot assign requested address',))
suthost_1   | 
suthost_1   | During handling of the above exception, another exception occurred:
suthost_1   | 
suthost_1   | Traceback (most recent call last):
suthost_1   |   File "/workdir/service.py", line 19, in wiremock
suthost_1   |     response = requests.get("%s/get/this" % WIREMOCK_URL).text
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 72, in get
suthost_1   |     return request('get', url, params=params, **kwargs)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 58, in request
suthost_1   |     return session.request(method=method, url=url, **kwargs)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
suthost_1   |     resp = self.send(prep, **send_kwargs)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
suthost_1   |     r = adapter.send(request, **kwargs)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 508, in send
suthost_1   |     raise ConnectionError(e, request=request)
suthost_1   | requests.exceptions.ConnectionError: HTTPConnectionPool(host='wiremock', port=8080): Max retries exceeded with url: /get/this (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff3c7bcbfd0>: Failed to establish a new connection: [Errno 99] Cannot assign requested address',))
suthost_1   | Traceback (most recent call last):
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
suthost_1   |     (self.host, self.port), self.timeout, **extra_kw)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
suthost_1   |     raise err
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
suthost_1   |     sock.connect(sa)
suthost_1   | OSError: [Errno 99] Cannot assign requested address
suthost_1   | 
suthost_1   | During handling of the above exception, another exception occurred:
suthost_1   | 
suthost_1   | Traceback (most recent call last):
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
suthost_1   |     chunked=chunked)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
suthost_1   |     conn.request(method, url, **httplib_request_kw)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
suthost_1   |     self._send_request(method, url, body, headers, encode_chunked)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
suthost_1   |     self.endheaders(body, encode_chunked=encode_chunked)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
suthost_1   |     self._send_output(message_body, encode_chunked=encode_chunked)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 1026, in _send_output
suthost_1   |     self.send(msg)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 964, in send
suthost_1   |     self.connect()
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
suthost_1   |     conn = self._new_conn()
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
suthost_1   |     self, "Failed to establish a new connection: %s" % e)
suthost_1   | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff3c7bcb4e0>: Failed to establish a new connection: [Errno 99] Cannot assign requested address
suthost_1   | 
suthost_1   | During handling of the above exception, another exception occurred:
suthost_1   | 
suthost_1   | Traceback (most recent call last):
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
suthost_1   |     timeout=timeout
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
suthost_1   |     _stacktrace=sys.exc_info()[2])
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
suthost_1   |     raise MaxRetryError(_pool, url, error or ResponseError(cause))
suthost_1   | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='wiremock', port=8080): Max retries exceeded with url: /get/this (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff3c7bcb4e0>: Failed to establish a new connection: [Errno 99] Cannot assign requested address',))
suthost_1   | 
suthost_1   | During handling of the above exception, another exception occurred:
suthost_1   | 
suthost_1   | Traceback (most recent call last):
suthost_1   |   File "/workdir/service.py", line 19, in wiremock
suthost_1   |     response = requests.get("%s/get/this" % WIREMOCK_URL).text
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 72, in get
suthost_1   |     return request('get', url, params=params, **kwargs)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 58, in request
suthost_1   |     return session.request(method=method, url=url, **kwargs)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
suthost_1   |     resp = self.send(prep, **send_kwargs)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
suthost_1   |     r = adapter.send(request, **kwargs)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 508, in send
suthost_1   |     raise ConnectionError(e, request=request)
suthost_1   | requests.exceptions.ConnectionError: HTTPConnectionPool(host='wiremock', port=8080): Max retries exceeded with url: /get/this (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff3c7bcb4e0>: Failed to establish a new connection: [Errno 99] Cannot assign requested address',))
suthost_1   | Traceback (most recent call last):
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
suthost_1   |     (self.host, self.port), self.timeout, **extra_kw)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
suthost_1   |     raise err
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
suthost_1   |     sock.connect(sa)
suthost_1   | OSError: [Errno 99] Cannot assign requested address
suthost_1   | 
suthost_1   | During handling of the above exception, another exception occurred:
suthost_1   | 
suthost_1   | Traceback (most recent call last):
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
suthost_1   |     chunked=chunked)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
suthost_1   |     conn.request(method, url, **httplib_request_kw)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
suthost_1   |     self._send_request(method, url, body, headers, encode_chunked)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
suthost_1   |     self.endheaders(body, encode_chunked=encode_chunked)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
suthost_1   |     self._send_output(message_body, encode_chunked=encode_chunked)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 1026, in _send_output
suthost_1   |     self.send(msg)
suthost_1   |   File "/usr/local/lib/python3.6/http/client.py", line 964, in send
suthost_1   |     self.connect()
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
suthost_1   |     conn = self._new_conn()
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
suthost_1   |     self, "Failed to establish a new connection: %s" % e)
suthost_1   | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff3c7bea2e8>: Failed to establish a new connection: [Errno 99] Cannot assign requested address
suthost_1   | 
suthost_1   | During handling of the above exception, another exception occurred:
suthost_1   | 

<snip a lot of pages of the above>

suthost_1   | During handling of the above exception, another exception occurred:
suthost_1   | 
suthost_1   | Traceback (most recent call last):
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
suthost_1   |     timeout=timeout
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
suthost_1   |     _stacktrace=sys.exc_info()[2])
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
suthost_1   |     raise MaxRetryError(_pool, url, error or ResponseError(cause))
suthost_1   | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='wiremock', port=8080): Max retries exceeded with url: /get/this (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff3c74adb70>: Failed to establish a new connection: [Errno 99] Cannot assign requested address',))
suthost_1   | 
suthost_1   | During handling of the above exception, another exception occurred:
suthost_1   | 
suthost_1   | Traceback (most recent call last):
suthost_1   |   File "/workdir/service.py", line 19, in wiremock
suthost_1   |     response = requests.get("%s/get/this" % WIREMOCK_URL).text
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 72, in get
suthost_1   |     return request('get', url, params=params, **kwargs)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 58, in request
suthost_1   |     return session.request(method=method, url=url, **kwargs)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
suthost_1   |     resp = self.send(prep, **send_kwargs)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
suthost_1   |     r = adapter.send(request, **kwargs)
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 508, in send
suthost_1   |     raise ConnectionError(e, request=request)
suthost_1   | requests.exceptions.ConnectionError: HTTPConnectionPool(host='wiremock', port=8080): Max retries exceeded with url: /get/this (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff3c74adb70>: Failed to establish a new connection: [Errno 99] Cannot assign requested address',))
suthost_1   | *** Error in `/usr/local/bin/python3.6': free(): invalid pointer: 0x00000000018e7758 ***
suthost_1   | Fatal Python error: Aborted
suthost_1   | 
suthost_1   | Current thread 0x00007ff3cee96700 (most recent call first):
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/hooks.py", line 18 in default_hooks
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 295 in __init__
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 426 in prepare_request
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 494 in request
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 58 in request
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 72 in get
suthost_1   |   File "/workdir/service.py", line 19 in wiremock
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/japronto/app/__init__.py", line 182 in serve
suthost_1   |   File "/usr/local/lib/python3.6/multiprocessing/process.py", line 93 in run
suthost_1   |   File "/usr/local/lib/python3.6/multiprocessing/process.py", line 249 in _bootstrap
suthost_1   |   File "/usr/local/lib/python3.6/multiprocessing/popen_fork.py", line 74 in _launch
suthost_1   |   File "/usr/local/lib/python3.6/multiprocessing/popen_fork.py", line 20 in __init__
suthost_1   |   File "/usr/local/lib/python3.6/multiprocessing/context.py", line 277 in _Popen
suthost_1   |   File "/usr/local/lib/python3.6/multiprocessing/context.py", line 223 in _Popen
suthost_1   |   File "/usr/local/lib/python3.6/multiprocessing/process.py", line 105 in start
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/japronto/app/__init__.py", line 228 in _run
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/japronto/app/__init__.py", line 264 in run
suthost_1   |   File "/workdir/service.py", line 27 in <module>
suthost_1   |   File "/usr/local/lib/python3.6/runpy.py", line 85 in _run_code
suthost_1   |   File "/usr/local/lib/python3.6/runpy.py", line 96 in _run_module_code
suthost_1   |   File "/usr/local/lib/python3.6/runpy.py", line 263 in run_path
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/japronto/runner.py", line 77 in run
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/japronto/__main__.py", line 26 in main
suthost_1   |   File "/usr/local/lib/python3.6/site-packages/japronto/__main__.py", line 29 in <module>
suthost_1   |   File "<frozen importlib._bootstrap>", line 205 in _call_with_frames_removed
suthost_1   |   File "<frozen importlib._bootstrap_external>", line 678 in exec_module
suthost_1   |   File "<frozen importlib._bootstrap>", line 655 in _load_unlocked
suthost_1   |   File "<frozen importlib._bootstrap>", line 950 in _find_and_load_unlocked
suthost_1   |   File "<frozen importlib._bootstrap>", line 961 in _find_and_load
suthost_1   |   File "/usr/local/bin/japronto", line 7 in <module>
suthost_1   | Worker crashed on signal SIGABRT!
suthost_1   | Worker crashed on signal SIGSEGV!
suthost_1   | Worker crashed on signal SIGSEGV!
suthost_1   | Worker crashed on signal SIGSEGV!
suthost_1   | Worker crashed on signal SIGABRT!
suthost_1   | Worker crashed on signal SIGSEGV!
suthost_1   | Worker crashed on signal SIGSEGV!
suthost_1   | Worker crashed on signal SIGSEGV!
suthost_1   | Worker crashed on signal SIGSEGV!
suthost_1   | Worker crashed on signal SIGSEGV!
suthost_1   | Worker crashed on signal SIGABRT!
suthost_1   | Worker crashed on signal SIGABRT!
suthost_1   | Worker crashed on signal SIGABRT!
suthost_1   | Worker crashed on signal SIGSEGV!
suthost_1   | Worker crashed on signal SIGSEGV!

What could be the reason for this? (memory leak, or something else?)

tejastank commented 6 years ago

Japronto yet not stable or somewhere active productive things, Don't think to use for production. yet pending issue and similar your reported issues we faced too.

vadim-shadrin commented 6 years ago

Let's make japronto stable so that it can be used for production. Honestly, I already use it and I hope that the author will be able to fix bugs with our help.

tejastank commented 6 years ago

We also interested to use it, but don’t trust until dedication brings on open version.

mcauto commented 6 years ago

@carniz Could you check file description count of suthost_1 container? I think a reason is socket reuse

oranb83 commented 6 years ago

@squeaky-pl I'm using Japronto in production, but when I changed my code to work with more than one worker_num, I started to memory leak issues. Using one worker there is no issue even after months of work and I monitor the AWS instances very carefully via CloudWatch. I really hope you can look into this issue. If you can create a PR I will test it and let you know if the issue has been resolved.