Discuss API-Design and Web-Framework (tide) alternatives

[NicolaLS]

I am currently re-implementing #55 and had issues with tide not supporting http 1.0 (tides dependency respectively) which is needed for the json-rpc client. Now I am wondering if we should migrate away from tide and use a different web-framework. Since I'm new to web-dev I really don't know how to choose one over another so a discussion about that would be great. My only criteria would be that it's lightweight (?) @elsirion said that somebody already suggested axum.

Also since I am re-implementing the API anyway this would be a chance to discuss general decisions and the API standard to choose. I wanted to go for JSON-2.0 RPC but feel free to discuss here if something else would be better (or multiple APIs make sense). Feel free to comment your thoughts (not only from a dev decision perspective but maybe also a API consumer (e.g app @justinmoon @Maan2003 )

[justinmoon]

Tide also doesn't seems to have a good authentication library. The best option I've seen hasn't had a commit in 1 year. I think we should switch to something else, but haven't investigated yet. Have also heard good things about Axum.

In terms of consuming the API, what are the intended usecases for clientd? I think mobile apps will connect directly to federation / gateway servers.

[NicolaLS]

I looked into actix-web, warp, axum and hyper now. Would it be ok to just go for hyper ? Some code that uses tide right now would be inflated a lot though I guess. If not hyper, I'd think axum is a good choice.

Good question, I thought that the app would use it but thinking about it I see why not. This started as a "first good issue" for me. I guess the goel was to just have a more convenient client. So maybe clientd + cli is only for administrators/devs ? Maybe also for people who want to build a project interacting with minimint so they can have a simple API to build/test their stuff first and than move to connecting to the fed/gateway themselves.

[matthiasdebernardini]

@NicolaLS hyper is a (very) solid http library, using it for the client is a good idea. I like its correctness guarantees and that its acknowledged by large projects to be of high quality (curl project). I am also not sure what the minimint client is supposed to be other than a mobile app running on low-end hardware in a spotty network, this will not be running a webserver with state, so a web framework wouldn't be needed. The client may not need to be async as well depending on how interactive the client-server protocol is (I still haven't seen it). If you go to the tokio website you'll see how all the crates stack up.

Hyper and axum are both written by the same team that wrote the tokio runtime. The benefit of this is that they can more coherently layout the types as they relate to the separate components of the web stack (not that actix isn't). The question is if they wrote axum for people to use in production or if they made it to show off the tokio ecosystem. Due to the large backing, I think its safe to assume that it will be around for some time. Some benefits of axum is that it uses tokio runtime (tide uses async-std, though I think ours is configured differently), which is IMHO where the work will accumulate (rustaceans will prefer to implement the tokio traits for tokio types rather than any alternative).

Async-std is not part of the standard library and it never will be, it has the name "std" in it because there is no name control in the ecosystem.

We should absolutely move away from tide or the fedimint project will end up solving the "how to make web api ecosystem in rust" problem rather than the "how do we get bitcoin into the hands of billions of people in a scalable and private manner" problem - which is actually worth solving.

Below is a blog post I read that affirmed my preference (I also have his book, so far its good).

https://kerkour.com/rust-web-framework-2022

Axum is (kinda) the winner except for very large projects that need unmatched performance (actix would be the pick for that). I don't think that raw performance is top priority for fedimint (axum claims to be ergonomic, which I personally value). The lack of maturity hurts in battle testing, documentation and feature availability - all of which are improving (could be us as well) every day the same way rust is. I don't think correctness is impacted because all the other parts that axum uses are more mature and more thoroughly tested (tokio, bytes, hyper and tower).

The author of the blog also recommends that we use a "clean architecture" for our API so that we can swap it out in the future (sounds like good advice). Maybe in the future there will be a huge fedimint and the federation will end up offering lots and lots of services (media, messaging, oracles, etc) to millions of people. So having a way to swap out frameworks for those that need it would be good.

I wrote (copied and pasted) an axum web API for a personal project (uploads public keys and matches them with valid signatures and the data) and I liked it, everything worked the way I expected it to.

I should note that it is a basic service, so I haven't looked at all the ways to use axum nor can I confidently comment on how well suited axum is for fedimint. My endorsement and opinion is from the perspective of an enthusiast or hobbyist (though hopefully not for much longer).

If I had to make a choice it would be axum or actix.

Web API's are commodities at this point, I think we should go with whatever seems to be the easiest.

I'm still getting familiar with the architecture and code to decide on what we should use as an interface. So I can't comment on whether to use an RPC interface or a web API interface.

[NicolaLS]

Thank you for the extensive answer @matthiasdebernardini, helps a lot ! I really like that axum is "from" tokio and even tough I have no real reason for it I think I'd prefer it over actix. I'll just stay with JSON 2.0 RPC then since it's already (almost) done but I don't think adding another API would be too much work if its wanted in the future. So @elsirion would it be fine for you to move to axum from tide ? Should this be the "task" issue or should I close this and open a "migrate from tide to axum" one ?

[matthiasdebernardini]

Happy I could help!

@NicolaLS For the API, which components will be using the RPC interface? If it's between a client and a user then I would think that a RESTful interface is all that is needed.

[NicolaLS]

Happy I could help!

@NicolaLS For the API, which components will be using the RPC interface? If it's between a client and a user then I would think that a RESTful interface is all that is needed.

mhm..yes I think so. While thinking about how to re-implement the jsonrpc I notice also more and more hurdles in comparison to RESTful. RESTful it is then for now :+1:

[elsirion]

So @elsirion would it be fine for you to move to axum from tide

Sorry for the late reply, I think if we can get rid of dependencies that way it's a good thing. Axum seems stable enough. Feel free to open a separate issue if you think there's more to discuss that would be buried here. Otherwise just keeping this and opening a PR is also fine :)

[justinmoon]

@matthiasdebernardini is going to attempt the Axum switch

[maan2003]

I am working on this part of codebase for #195 I would like to work on this.

[NicolaLS]

I thought in Rust dev we don't have race conditions 😂

[justinmoon]

@matthiasdebernardini do you mind helping review @NicolaLS's PR instead?

[NicolaLS]

I think we agreed to use axum instead of tide

[elsirion]

Is this actually still relevant after #203 ?

[justinmoon]

Is this actually still relevant after #203 ?

Gateway still uses tide, and nico has a PR to switch that to Axum.