Dusk-Labs / dim

Dim, a media manager fueled by dark forces.
GNU Affero General Public License v3.0
3.8k stars 160 forks source link

Improve rebuild time when no back end code has changed #381

Closed jimmycuadra closed 10 months ago

jimmycuadra commented 2 years ago

If you are running the development server, stop it, and then start it again (via cargo run) without any changes to Rust source code, it should run the existing executable immediately. Instead, it takes a while (roughly 30 seconds on my machine) to rebuild, showing the following output while waiting:

$ cargo run
   Compiling dim v0.3.0-rc6 (/path/to/dim/dim)
warning: `ui/build` does not exist.
warning: If you wish to embed the webui, run `yarn build` in `ui`.
   Compiling database v0.1.0 (/path/to/dim/database)
warning: Generating "./dim_dev.db" from latest migrations.
warning: Built database /path/to/dim/./dim_dev.db.
    Finished dev [unoptimized + debuginfo] target(s) in 31.31s

It looks like this is happening because migrations are being run unconditionally during the build. It would make development faster if the server could be quickly stopped and restarted without paying this penalty every time.

vgarleanu commented 2 years ago

I wonder if we can detect whether we need to run migrations or not with the migrations api that we're using in build.rs.

ta-vroom commented 2 years ago

I am pretty sure it's because of sqlx macros. I noticed it on another project, but I have not yet finished replacing the macros to see if it makes a difference.

From launchbadge/sqlx:

We can use the macro, sqlx::query! to achieve compile-time syntactic and semantic verification of the SQL, with an output to an anonymous record type where each SQL column is a Rust field (using raw identifiers where needed).

cobyge commented 2 years ago

Warp also takes a huge amount of time to compile, as each function which returns impl Filter<Extract = impl warp::Reply, Error = Rejection> + Clone adds to monomorphization time. Returning a BoxedFilter<Something> instead would reduce compile times, as Rust wouldn't need to know the size of the return value.

vgarleanu commented 2 years ago

@cobyge your suggestion has the downside that we will be boxing a lot of routes. What I would prefer instead is maybe we can group certain routes together and depending if we are running in debug mode or not, we can box them. Imo this would be cleaner than moving everything to use BoxedFilter.

On the note of monomorphization, I wonder if enabling -Z share-generics=y would speed things up. It's still unstable, but we could at the very least use this to speed up dev-build time.

cobyge commented 2 years ago

Doesn't seem like share-generics would help here, as that only lets crates share monomorphizations, which isn't the issue here. Why is boxing routes an issue? The only issue I can see with boxing routes, is the "performance" we'd lose by using dynamic dispatch, but I honestly doubt that will actually have an end-user impact.

Another alternative could possibly be a macro around the return type, which would resolve to a BoxedFilter on Debug, and the generic type on Release.