Performance: Spike

[mattmcgrattan]

Questions to consider:

• [ ] Assumption: Elasticsearch is the best "database" to use.
• [ ] Python web frameworks, we know that Flask is not very fast, but is it fast enough?
  • [ ] Flask (https://github.com/pallets/flask)
  • [ ] Falcon (https://github.com/falconry/falcon)
  • [ ] Hug (https://github.com/timothycrosley/hug)
  • [ ] Aiohttp (as a server, not client - https://github.com/aio-libs/aiohttp#server)
  • [ ] Starlette (https://www.starlette.io/)
• [ ] Interpreters:
  • [ ] CPython
  • [ ] Cython
  • [ ] PyPy
• [ ] Elasticsearch best practice.

Notes:

https://www.techempower.com/benchmarks/#section=data-r17&hw=ph&test=fortune&l=zijzen-1 https://fgimian.github.io/blog/2018/05/17/choosing-a-fast-python-api-framework/

[mattmcgrattan]

FWIW, my own current assumption is that probably Cython is a stretch too far in terms of maintainability, and getting Python code where a high percentage of the code is rendered as native C by cythonize is tricky. Often it's a mix of C and Python, which works fine, but you lose some of the speed advantages of native compilation.

Testing with CPython 3.x and PyPy from the start would be pretty useful, though. PyPy's JIT compiler is really pretty good, and generally "just works" with most standard Python code and libraries.

[mattmcgrattan]

We also haven't set yet what we consider adequate performance, in terms of response time, number of concurrent requests, etc.

[stephenwf]

Also related:

• logging performance
• idle-load overhead
• queue persistence

Although the last point may be its own ticket

[john-root]

• [ ] Python web frameworks, we know that Flask is not very fast, but is it fast enough?

  • [ ] Flask (https://github.com/pallets/flask)
  • [ ] Falcon (https://github.com/falconry/falcon)
  • [ ] Hug (https://github.com/timothycrosley/hug)
  • [ ] Aiohttp (as a server, not client - https://github.com/aio-libs/aiohttp#server)
  • [ ] Starlette (https://www.starlette.io/)
  • [ ] Interpreters:

The advantages of Flask include: currently widely adopted within the DLCS (commonality, consistency) commonly used so more likely that a given Python developer has experience many available modules for common requirements

(Django also meets all the above too, though I wouldn't personally want to use it for this)

[john-root]

Testing with CPython 3.x and PyPy from the start would be pretty useful, though. PyPy's JIT compiler is really pretty good, and generally "just works" with most standard Python code and libraries.

I think this makes sense and having it working with PyPy seems a realistic and useful goal