noirbizarre / flask-restplus

Fully featured framework for fast, easy and documented API development with Flask
http://flask-restplus.readthedocs.org
Other
2.73k stars 507 forks source link

Optimize performances #46

Open noirbizarre opened 9 years ago

noirbizarre commented 9 years ago

Some things make API instanciation very longs (seen in testing on some projects)

deepcopy usage cost way too much.

davidbgk commented 9 years ago

You may want to implement __deepcopy__ to speed up the process given http://stackoverflow.com/questions/24756712/deepcopy-is-extremely-slowir write your own deepcopy…

frol commented 9 years ago

You may consider using Immutable Data structures instead of copying all data. The best fitting project I know (maintained and feature-rich) is Pyrsistent: http://pyrsistent.readthedocs.org/en/latest/intro.html

(Just for the reference, I also came across other Python projects implemented Immutable Data structures, but, unfortunately, they are all abandoned: funktown, pysistence, fn.py)

noirbizarre commented 9 years ago

It can be an approach but to be, the main problem is caused by the flask-restful dependency: we started to differ too much and I have to do deepcopy to maintain compatibility with upstream marshalling algorithm.

Flask-restful was a very good starter for me but I think time has come to drop flask-restful and to make flask-restplus a standalone extension.

Qwasser commented 7 years ago

Hello! I updated to newest version from 0.7.2 and noticed that performance degraded significantly. Test script:

import cProfile, pstats, StringIO

from flask import Flask
import flask_restplus_new  # 0.10.1
import flask_restplus_old  # 0.7.2

def make_app(flask_resplust):
    app = Flask(__name__)

    api = flask_resplust.Api(app)

    client = api.model('Client', {
        'id': flask_resplust.fields.Integer(min=0, required=True, description='The client identifier'),
        'name': flask_resplust.fields.String(required=True, description='The client name')
    })

    namespace = api.namespace('clients', description='Client operations')

    @namespace.route('/<string:check_id>')
    class Check(flask_resplust.Resource):
        @api.marshal_with(client)
        def get(self, check_id):
            return {
                'id': 100500,
                'name': "test"
            }
    return app

def test_app(app):
    client = app.test_client()

    pr = cProfile.Profile()
    pr.enable()
    for i in xrange(5000):
        client.get('/clients/{}'.format(i))
    pr.disable()
    s = StringIO.StringIO()
    ps = pstats.Stats(pr, stream=s).sort_stats('tottime')
    ps.print_stats(.02)
    print s.getvalue()

if __name__ == "__main__":
    old_app = make_app(flask_restplus_old)
    test_app(old_app)

    new_app = make_app(flask_restplus_new)
    test_app(new_app)

Resulting timings are 6.986 seconds for 0.7.2 and 8.756 seconds for 0.10.1. There are some cases where performance degradation is more severe. One of the reasons is usage of deepcopy. Is it possible to fix this performance issue?

JordanP commented 7 years ago

Is it possible to fix this performance issue?

It might be possible. @Qwasser If you have some code optimization to share, I'll be happy to review your pull request.