search

zacharygolba / via

A multi-threaded async web framework for people who appreciate simplicity.
Apache License 2.0
1 stars 0 forks source link

Re-write and re-imaging of via #3

Closed zacharygolba closed 3 weeks ago

zacharygolba commented 4 months ago

Background

I started this project about four years ago as an experiment in exploring different code reuse patterns and separating concerns in Rust. At the time, async Rust was still in its infancy, and this project helped me understand the performance characteristics and constraints associated with concurrent programming in Rust. After putting together a few examples inspired by other web application frameworks with a strong OO influence, I eventually shelved the project for a couple of years.

Fast forward to June of this year. I found myself with some spare time and decided to dive back into Rust. What better way to reacquaint myself with the language than by revisiting an old project, right? Returning to the project with fresh eyes, I realized my original approach to thinking about Rust code was not very... rusty. This led me back to the drawing board to reassess my goals. With a background in the agency world and experience in both backend and frontend projects, I wanted to create something I would have appreciated in past projects. For better or worse, this brought me back to thinking about JavaScript.

JavaScript has been a popular choice for many high-profile clients at the agency I previously worked for. Being a multi-paradigm language, JavaScript allows programmers to express themselves in many ways. Combining its good parts with a relatively quick learning curve, it becomes a reliable choice for companies aiming for productivity. This got me thinking: since Rust is also a multi-paradigm language with an expressive type system and powerful features, is there a way to combine simplicity, power, and expressiveness to make Rust the best choice for backend server development?

Async Rust has evolved significantly over the past four years. Like JavaScript, it has both its strengths and weaknesses. Fortunately, many great frameworks now make it enjoyable to be a Rust programmer. If your team knows what they're doing, it's hard to go wrong with the available frameworks today. With that said, I envisioned a framework that caters to both Rust experts and beginners. Here is what I came up with.

Principles

At the start of any project, it's essential to define the problem you're solving and decide on the trade-offs you're willing to make. We've seen time and again that file systems, databases, and external services often become the bottlenecks in a web application backend. However, we're writing Rust for a reason, so let's not be wasteful with CPU cycles or memory where it matters. But let's also be practical and think about the people writing the code rather than just the computer running it.

Hyper is a state-of-the-art HTTP library that offers excellent performance out of the box. Like many other frameworks, starting with a solid foundation, such as a fast and correct HTTP implementation, is crucial. However, what happens next can make or break the rest of the stack.

Consider the following principles like layers of a cake, each representing a different level of abstraction. Starting from the bottom up. Except for the section on Security—it's mentioned last but it's not the top layer. It's more like the plate that the cake is served on.

Fast and Flexible Routing

One hallmark of a great web application backend is the separation of concerns. When onboarding to a new project, a great way to start is by examining all the routes available in an application. URL paths and routes often form a tree-like structure that indicates the separation of concerns for a particular web server. For example, routes prefixed with /api will likely return a JSON response. With that in mind, let's use the path as a starting point for code reuse. Since the router I started working on four years ago could be considered a prefix-tree, we were already off to a good start.

Philosophy

An engineer should be able to open a file and see every possible route that can serve a request. Co-locating associated middleware for a route and its descendants alongside the HTTP method a route accepts makes it easy to see what happens when someone visits the corresponding URL. Let's not introduce any ambiguity here. In my opinion, it's more important to see the HTTP method at the route definition rather than the function. Additionally, it's helpful to know that the function signature for any given middleware or responder is async fn _(request: Request, next: Next) -> Result<Response>. Therefore, I decided to remove the codegen crate to rule out the use of proc-macros to define routes.

Many frameworks, such as Tide, follow this philosophy, and you can see Tide's influence on the public API of the router. However, there's one crucial difference in how routing works in via compared to Tide. Via supports the concept of partial route matches by leveraging the tree-like data structure of our routes. The benefit of this feature is that we can use the URL path as a 1-to-1 mapping of the separation of concerns across routes and resources in a via application. For example, suppose a new JIRA ticket is automatically created from webhooks to an error reporting service with the title JSON.parse: unexpected character at line 1 column 1 of the JSON data. This is likely caused by an error response returned from an endpoint that should respond with JSON, but in case of an error, a plaintext response is sent to the client. With via, you can include middleware that runs unconditionally anytime the URL path partially matches a route. This means you can apply error-handling logic, compression, authentication for any middleware or responder downstream from /api without having to repeat yourself or introduce some higher-order function that eventually becomes the source of every heisenbug in your app. The benefit of defining routes this way can be seen in the blog-api example.

let mut api = app.at("/api");

// Apply specific error handling logic to the /api namespace.
api.include(ErrorBoundary::map(|mut error| {
    use diesel::result::Error as DieselError;

    if let Some(DieselError::NotFound) = error.source().downcast_ref() {
        // The error occurred because a record was not found in the
        // database, set the status to 404 Not Found.
        *error.status_mut() = StatusCode::NOT_FOUND;
    }

    // Return the error with the response format of JSON.
    error.json()
}));

api.at("/posts").scope(|posts| {
    use api::posts;

    posts.include(posts::authenticate);

    posts.respond(via::get(posts::index));
    posts.respond(via::post(posts::create));

    posts.at("/:id").scope(|post| {
        post.respond(via::get(posts::show));
        post.respond(via::patch(posts::update));
        post.respond(via::delete(posts::destroy));
    });
});

api.at("/users").scope(|users| {
    use api::users;

    users.respond(via::get(users::index));
    users.respond(via::post(users::create));

    users.at("/:id").scope(|user| {
        user.respond(via::get(users::show));
        user.respond(via::patch(users::update));
        user.respond(via::delete(users::destroy));
    });
});

Performance

Now that we've defined how we want the router to work, the next step was to build it and make it fast. The result of the time spent optimizing the router when I started this rewrite a month ago can be seen by running the benchmark in via-router on this branch. On my machine, it takes roughly 135ns per iteration to find all matching routes for a path with nested resources in a router with 100 routes. This is better than I anticipated and fast enough to allow for some compromises higher up in the stack.

Pragmatic Trade-Offs

We all have fought with the borrow checker and know how it feels after a long day of helpful yet despairing compiler errors. Often, the solution to these errors involves re-imagining and re-ordering the sequence of events in a function in a way that makes sense to a computer. This is an important part of being a Rust programmer. Nobody denies that. However, it's a hard conversation to have with your manager when you spend all day fighting the borrow checker, only to find

out that the implementation using borrowed data is slower than the version you initially wrote using Arc. It's also no secret that using borrows unnecessarily only makes it more difficult to onboard a Rust beginner. Still, this is a complex subject in the language, and we shouldn't hide the parts of Rust that one will inevitably have to use.

The Middle Ground

With via, I think it's important to find the intersection of performance and ease of use. Therefore, when it comes to data in a request, we treat performance requirements as a top priority. This means that when you work with path and query parameters from a via::Request, we return references (with the underlying type &str) that truly reference the data in the URL path. When you read the request body in a particular responder, the data is moved out of the request rather than copied to avoid wasting resources downstream. Above all, query parameters and other types of data related to a request (cookies, etc.) are always parsed and stored lazily so the query parser code does not run unless you actually use query parameters.

However, we don't want you to stay late at work unless you really, really want to. Therefore, we do some things that might be considered controversial by some in the Rust community (I'm half-joking here). We wrap middleware and responders added to a route in an Arc and use an owned reference to call the middleware when we unwind the stack because I like using the termination operator on line 1 of the function I'm writing. In all seriousness, I think it's important for productivity and onboarding to allow for request and next to move into the future returned from middleware. I believe the performance implications of doing so are acceptable for anyone comfortable using a framework rather than writing a hyper service from scratch for their next project. We also return a reference to an Arc of the global state returned from Request::state so you can clone it as much as you want and tell your boss that we told you to do it!

Unix-Like

When I initially started this four years ago, I wanted to provide a batteries-included option (similar to Django and Rails) but for Rust. However, I think it's important to support an ecosystem of libraries as unique as the community for which this framework was built. It's fun to write your own CORS middleware or static server, and we don't want to take that away from you. Via is meant to be a thin layer on top of hyper, giving you control over the important decisions. We'll provide a few "official" crates, such as via-serve-static introduced in this pull request, but we won't require you to use it or make it a part of our core API. Write your own if that makes you happy. That's what I told myself when I was looking at Rust web frameworks, and had I not done that, I would have been so bored last month. Competition is good for everyone, and I want to see if someone else will write an auth crate so I can deny culpability if need be. We'll also feature flag the stuff you expect, such as JSON support via serde, and never include default features.

* via-serve-static is very, very fast and secure, and is a great choice for serving static files, but you can still write your own if you want to.*

Security

Rust allows us to write code that is verified to be correct at compile time. I believe it is our responsibility to uphold the guarantees advertised by the language and not introduce unsafe blocks for a slight or even substantial performance improvement. I'm okay with being second best (or even in the top 5) if that means it's easier for me to sleep at night. I know what it's like to be a victim, and I love that Rust prioritizes safety and security.

The only place you should see unsafe in the source code is for Pin projections, as I believe it is important for us to write our own pin projections to verify their correctness as a community.

Epilogue

Thank you for taking the time to read this description. I know test coverage can be improved, and that's a priority of mine, but I appreciate your time and effort in reviewing this substantial PR. If anyone can confirm the existence of and/or pinpoint the source of a small but potential memory leak, I promise you will be the first to receive a via t-shirt if this framework becomes popular. I welcome your constructive feedback and insights. Let's work together to make this project reach the potential that I believe it has!

You can find more traditional examples in this work-in-progress branch: https://github.com/zacharygolba/via/tree/refactor-simplified-examples/docs/examples.

Stay tuned as I plan on adding some issues for features that didn't make it into this pull request over the next week.

Edit: From what I can tell, the memory leak in via-router has been fixed. There may be improvements to be made with regards to memory management but memory usage should be stable at this point.

Short-Term and Medium-Term Roadmap

Blocking the first release:

Nice to have for the first release:

Things to follow shortly after the first release:

zacharygolba commented 4 months ago

575f518 is my way of saying that I'm sorry (yes, I'm talking about at! and visit! 🤣). It is a bit verbose, but I think it is a good starting point to ensure that the code in via-router does what it is supposed to do. I'm also hopeful that it inspires the community to further improve the performance of the router.

zacharygolba commented 2 months ago

Considering the progress made in #11 and #9, here is short summary of what is preventing this PR from merging and the first version of Via from being released.

Things to follow shortly after the first release:

zacharygolba commented 2 months ago

If you want to try out Via and want to know what branch you should specify in your Cargo.toml, use refactor-app-service for now.

Examples: https://github.com/zacharygolba/via/tree/refactor-app-service/docs/examples

Sorry in advance for the empty error-handling example. Check out main.rs of the blog-api example (formerly know as advanced-blog) if your curious to see how error handling works.

zacharygolba commented 2 months ago

I'm considering refactoring the error module and replacing the custom type with an extension trait that works seamlessly with std::error::Error and also serde::Serialize if the json feature flag is enabled.

zacharygolba commented 4 weeks ago

The FlatMap task could be a Fold. However, I like the idea of FlatMap since the input and output type would always be Bytes. We could borrow some of the logic in the implementation of Buffered to ensure that whatever is returned from the map fn will be flattened to 8KB chunks. I do like the idea of supporting local state for the map function similar to the accumulator argument passed to fold. In my opinion, that would allow the vast majority of use-cases to benefit from a statically-dispatched body. For all other use-cases, the http-body-util crate can be used in combination with AnyBody::Box(BoxBody). Once implemented, the AnyBody enum can be marked as non_exhaustive.