Consider migration from CherryPy to Flask for WMCore

[vkuznet]

Recently @vkuznet implemented support of x509-scitokens [1] and SSO [2] for cmsweb frontend. Both of them relies on using Flask+WSGI for cmsweb. Also @vkuznet presented concurrency studies [3] which shows that usage of Flask+WSGI may speed-up python web-based frameworks up to 30%. This brings the question: should we consider a migration of WMCore+CherryPy stack to WMCore+Flask+WSGI one. A proof-of-concept for DBS server used in [3] studies can be found here [4]. In particular, @vkuznet used Flask with DBS server, see lines 455-519 for Flask specific code. It also provides an example of dealing with DB connection and setting up pool of dbs agents. The Flask code basically wrap existing class into series of end-point and if we have clear separation of business and web logic the migration can be done easily.

I suggest to do evaluation of new technology and may be use spring, summer students to work on providing migration from CherryPy to Flask technology.

Best, Valentin.

[1] https://github.com/dmwm/deployment/pull/706 [2] https://github.com/dmwm/deployment/pull/711 [3] https://indico.cern.ch/event/705389/contributions/2894815/attachments/1599683/2535722/WebConcurrencyStudies.pdf [4] https://gist.github.com/vkuznet/70afa8aa2a8af314a3ecae6796c569b2

[amaltaro]

Thanks for these links, Valentin. I'll look into it later this week

[amaltaro]

I've just seen this article comparing Flask with FastAPI: https://www.turing.com/kb/fastapi-vs-flask-a-detailed-comparison

A few things are not clear to me, but I think it is worth it to share it.

In addition, Unified frontend has been (re-)implemented in FastAPI, so @hassan11196 would be a good person to talk to about experience with FastAPI, eventually.

[vkuznet]

I suggest to pay attention to the following characteristics:

• benchmark performance, e.g. Go vs Python https://www.techempower.com/benchmarks/#hw=ph&test=fortune&section=data-r22&l=zijmrj-cn3 or choose your filter accordingly

• dependencies, e.g.

  
  python -m venv venv
  source venv/bin/activate
  pip install "fastapi[standard]"
  ...
  Installing collected packages: websockets, uvloop, typing-extensions, sniffio, shellingham, pyyaml, python-multipart, python-dotenv, pygments, mdurl, MarkupSafe, idna, httptools, h11, dnspython, click, certifi, annotated-types, uvicorn, pydantic-core, markdown-it-py, jinja2, httpcore, email-validator, anyio, watchfiles, starlette, rich, pydantic, httpx, typer, fastapi, fastapi-cli
  Successfully installed MarkupSafe-3.0.2 annotated-types-0.7.0 anyio-4.6.2.post1 certifi-2024.8.30 click-8.1.7 dnspython-2.7.0 email-validator-2.2.0 fastapi-0.115.3 fastapi-cli-0.0.5 h11-0.14.0 httpcore-1.0.6 httptools-0.6.4 httpx-0.27.2 idna-3.10 jinja2-3.1.4 markdown-it-py-3.0.0 mdurl-0.1.2 pydantic-2.9.2 pydantic-core-2.23.4 pygments-2.18.0 python-dotenv-1.0.1 python-multipart-0.0.12 pyyaml-6.0.2 rich-13.9.3 shellingham-1.5.4 sniffio-1.3.1 starlette-0.41.0 typer-0.12.5 typing-extensions-4.12.2 uvicorn-0.32.0 uvloop-0.21.0 watchfiles-0.24.0 websockets-13.1

ls venv/lib/python3.11/site-packages | wc -l 75

- benefits of framework, e.g. if your framework provides X throughput but your business logic handles only Y where `Y >> X` your entire throughput will be driven by Y rather than X. As such a simple switch of a web framework may not be beneficial if underlying app logic will not be improved.