Look into automating much of API building

[mhauru]

There's a lot of repetitive code and documentation for the API. Almost all of the following things are done for almost all of the endpoints: Define the JSON schema of the input payload, define the JSON schema of the return payload, which status codes should be returned in which cases, what should the error messages be if the input is malformed, convert between JSON and python object, document this in the function docstring and in the API docs in backend/REAMDE.md.

There must be tools for automating this, so that we could just write the JSON schema definitions and some error responses in a concise format, and the tool would turn that into an endpoint. Look into this, and if we find a good tool, use it, otherwise maybe write our own automation.

[mhauru]

Turns out there's a plethora of tools for this. The key thing is the OpenAPI standard, and there are several tools for specifying your API in this standard and having the code and the docs generated, or for reading this standard from your API code and having docs generated. Selecting from the tools available, I would prefer one that

• is a Flask extension, rather than a whole framework in itself, even one built on Flask. This would rule out Connexion and APIFlask.
• supports OpenAPI 3.0. This would rule out Flask-RESTX and maybe Flasgger, for which support is "experimental". Might still try Flasgger, not the end of the world if we only support OpenAPI 2.0 (aka Swagger).

One that survives both of those filters is apispec, but it doesn't seem as fully featured and I'm not sure Flask-pluggability is as good, although I might be wrong on the latter. There's also flask-apispec, a Flask extension that uses apispec, but that's unmaintained and doesn't support OpenAPI 3.0.

flask-apispec also uses Marshmallow for serializing Python objects to JSON, and webargs for parsing webargs. We should probably start using webargs, it seems really handy. Maybe on the frontend too.

When you read about this, a common recommendation is to switch from Flask to FastAPI. Might be annoyingly big of an effort and not worth the switching cost, but worth keeping in mind as an option, it could also work out quite cleanly.

[mhauru]

I spent some time reading on switching to FastAPI, trying to decide if it would be worth it.

Benefits:

• Faster.
• Automated API docs in OpenAPI 3.0 with a Swagger UI page out of the box.
• Automated validation of JSON inputs out of the box, using the nice Pydantic style that we are trialing with models in #189.
• More modern, but hugely popular and thus presumably reliable and maintained into the future.

Cons:

• Cost of switching.
• More modern, meaning less of an ecosystem around it, although growing fast.
• No extensions/plugins. Would need to manually implement stuff to replace some Flask plugins we use. I think the only one of real effort would be the JWT authentication stuff. Nice tutorial for how to do it here: https://fastapi.tiangolo.com/tutorial/security/oauth2-jwt/

See here for a guide to transitioning from Flask to FastAPI: https://engineering.forethought.ai/blog/2022/12/01/migrating-from-flask-to-fastapi-part-1/#

For the cost of switching, I think transforming most of the API code would be very easy and done in an afternoon. The two parts that give me pause are the JWT authentication, and possibly some of the testing and pytest fixtures, although the latter could turn out to be trivially easy. We would also lose Flask-sqlalchemy, which I think we are only using to give a new database session for each API call and that would be very easy replace, and Flask-migration, which we import but I think don't use at all.

[mhauru]

Currently doing a little trial of how easily we could switch to FastAPI.

[EdwinB12]

Markus has been working on this for a few days. He has been blocked by a bug. If bug is not resolved within a few hours of coding time, then we will archive this for now.

[mhauru]

Got the bug cleared on Tuesday, finished all the hard parts, now most of the way there in doing the migration to FastAPI. Will finish next week.