search

ActualLab / Fusion

MIT License
76 stars 11 forks source link

readme

👾 Fusion: the "real-time on!" switch that actually exists

Build NuGet Version MIT License
Fusion Place Commit Activity Downloads

Overview

ActualLab.Fusion is a successor of Stl.Fusion - a distributed reactive memoization library for .NET that simplifies real-time updates, caching, and managing client-side state in complex distributed applications. By using dependency tracking and automated invalidation, Fusion ensures that values are recomputed only when necessary, making your application both efficient and responsive.

You can think of Fusion as:

Fusion reduces complexity for developers, allowing them to build scalable, real-time apps without the usual headaches associated with a set of notoriously difficult problems:

And the best part is: Fusion does all of that transparently for you, so Fusion-based code is almost identical to code that does not involve it.

Usage

To use Fusion, you must:

The magic happens when [ComputeMethod]-s are invoked:

  1. When Fusion knows that a value for a given call (think (serviceInstance, method, args...) cache key) is still consistent, Fusion returns it instantly, without letting the method to run.
  2. And when the value isn't cached or tagged as inconsistent, Fusion lets the method run, but captures new value's dependencies in process. "Dependency" is one [ComputeMethod] call triggered during the evaluation of another [ComputeMethod] call.

The second step allows Fusion to track which values are expected to change when one of them changes. It's quite similar to lot traceability, but implemented for arbitrary functions rather than manufacturing processes.

The last piece of a puzzle is Invalidation.Begin() block allowing to tag cached results as "inconsistent with the ground truth". Here is how you use it:

var avatars = await GetUserAvatars(userId);
using (Invalidation.Begin()) {
    // Any [ComputeMethod] invoked inside this block doesn't run normally,
    // but invalidates the result of the identical call instead.
    // Such calls complete synchronously and return completed Task<TResult>, 
    // so you don't need to await them.

    _ = userService.GetUser(userId);
    foreach (var avatar in avatars)
        _ = userAvatarService.GetAvatar(userId, avatar.Id);
}

The invalidation is always transitive: if GetUserProfile(3) calls GetUserAvatar("3:ava1"), and GetUserAvatar("3:ava1") gets invalidated, GetUserProfile(3) gets invalidated as well.

To make it work, Fusion maintains a dictionary-like structure that tracks every call result, where:

You can "pull" the Computed<T> instance "backing" certain call like this:

var computed1 = await Computed.Capture(() => GetUserProfile(3));
// You can await await for its invalidation:
await computed1.WhenInvalidated();
Assert.IsFalse(computed1.IsConsistent());
// And recompute it:
var computed2 = await computed1.Recompute();

So any Computed<T> is observable. Moreover, it can be a "replica" of a remote Computed<T> instance that mirrors its state in your local process, so the dependency graph can be distributed. To make it work, Fusion uses its own WebSocket-based RPC protocol, which is quite similar to any other RPC protocol:

  1. To "send" the call to a remote peer, client sends "call" message
  2. The peer responds to it with "call result" message. So far there is no difference with any other RPC protocol.
  3. And here is the unique step: the peer may later send a message telling that the call result it sent earlier was invalidated.

Step 3 doesn't change much in terms of network traffic: it's either zero or one extra message per call (i.e. 3 messages instead of 2 in the worst case). But this small addition allows Compute Service Clients to know precisely when a given cached call result becomes inconsistent.

The presence of step 3 makes a huge difference: any cached & still consistent result is as good as the data you'll get from the remote server, right? So it's totally fine to resolve a call that "hits" such a result locally, incurring no network round-trip!

Finally, any Compute Service Client behaves as a similar local Compute Service. Look at this code:

string GetUserName(id)
    => (await userService.GetUser(id)).Name;

You can't tell whether userService here is a local compute service or a compute service client, right?

So Fusion abstracts away the "placement" of a service, and does it much better than conventional RPC proxies: Fusion proxies aren't "chatty" by default!

Documentation

If you prefer slides, check out "Why real-time web apps need Blazor and Fusion?" talk - it explains how many problems we tackle are connected, how Fusion addresses the root cause, and how to code a simplified version of Fusion's key abstraction in C#.

The slides are slightly outdated - e.g. now Fusion clients use ActualLab.Rpc rather than HTTP to communicate with the server, but all the concepts they cover are still intact.

Quick Start, Cheat Sheet, and the Tutorial are the best places to start from.

Check out Samples; some of them are covered further in this document.

"What is your evidence?"*

All of this sounds way too good to be true, right? That's why there are lots of visual proofs in the remaining part of this document. But if you'll find anything concerning in Fusion's source code or samples, please feel free to grill us with questions @ Fusion Place!

Let's start with some big guns:

Check out Actual Chat – a very new chat app built by the minds behind Fusion.

Actual Chat fuses real-time audio, live transcription, and AI assistance to let you communicate with utmost efficiency. With clients for WebAssembly, iOS, Android, and Windows, it boasts nearly 100% code sharing across these platforms. Beyond real-time updates, several of its features, like offline mode, are powered by Fusion.

We're posting some code examples from Actual Chat codebase here, so join this chat to learn how we use it in a real app.

Now, the samples:

Below is Fusion+Blazor Sample delivering real-time updates to 3 browser windows:

Play with live version of this sample right now!

The sample supports both Blazor Server and Blazor WebAssembly hosting modes. And even if you use different modes in different windows, Fusion still keeps in sync literally every bit of a shared state there, including the sign-in state:

Is Fusion fast?

Yes, it's incredibly fast. Here is an RPC call duration distribution for one of the most frequent calls on Actual Chat:

IChats.GetTile reads a small "chat tile" - typically 5 entries pinned to a specific ID range, so it can be efficiently cached. And even for these calls the typical response time is barely measurable: every X axis mark is 10x larger than the previous one, so the highest peak you see is at 0.03ms!

The next bump at ~ 4-5ms is when the service actually goes to the DB - i.e. it's the time you'd expect to see without Fusion. The load would be way higher though, coz the calls you see on this chart are the calls which "made it" to the server - in other words, they weren't eliminated by the client / its Fusion services.

A small synthetic benchmark in Fusion test suite compares "raw" Entity Framework Core-based Data Access Layer (DAL) against its version relying on Fusion:

Calls/s PostgreSQL MariaDB SQL Server Sqlite
Single reader 1.02K 645.77 863.33 3.79K
960 readers (high concurrency) 12.96K 14.52K 16.66K 16.50K
Single reader + Fusion 9.54M 9.28M 9.05M 8.92M
960 readers + Fusion 145.95M 140.29M 137.70M 141.40M

The raw output for this test on Ryzen Threadripper 3960X is here. The number of readers looks crazy at first, but it is tweaked to maximize the output for non-Fusion version of DAL (the readers are asynchronous, so they mostly wait for DB response there).

Fusion's transparent caching ensures every API call result your code produces is cached, and moreover, even when such results are recomputed, they mostly use other cached dependencies instead of hitting a much slower storage (DB in this case).

And interestingly, even when there are no "layers" of dependencies (think only "layer zero" is there), Fusion manages to speed up the API calls this test runs by 8,000 to 12,000 times.

What makes Fusion fast:

Does Fusion scale?

Yes. Fusion does something similar to what any MMORPG game engine does: even though the complete game state is huge, it's still possible to run the game in real time for 1M+ players, because every player observes a tiny fraction of a complete game state, and thus all you need is to ensure the observed part of the state fits in RAM.

And that's exactly what Fusion does:

Check out "Scaling Fusion Services" part of the Tutorial to see a much more robust description of how Fusion scales.

Enough talk. Show me the code!

A typical Compute Service looks as follows:

public class ExampleService : IComputeService
{
    [ComputeMethod]
    public virtual async Task<string> GetValue(string key)
    { 
        // This method reads the data from non-Fusion "sources",
        // so it requires invalidation on write (see SetValue)
        return await File.ReadAllTextAsync(_prefix + key);
    }

    [ComputeMethod]
    public virtual async Task<string> GetPair(string key1, string key2)
    { 
        // This method uses only other [ComputeMethod]-s or static data,
        // thus it doesn't require invalidation on write
        var v1 = await GetNonFusionData(key1);
        var v2 = await GetNonFusionData(key2);
        return $"{v1}, {v2}";
    }

    public async Task SetValue(string key, string value)
    { 
        // This method changes the data read by GetValue and GetPair,
        // but since GetPair uses GetValue, it will be invalidated 
        // automatically once we invalidate GetValue.
        await File.WriteAllTextAsync(_prefix + key, value);
        using (Invalidation.Begin()) {
            // This is how you invalidate what's changed by this method.
            // Call arguments matter: you invalidate only a result of a 
            // call with matching arguments rather than every GetValue 
            // call result!
            _ = GetValue(key);
        }
    }
}

[ComputeMethod] indicates that every time you call this method, its result is "backed" by Computed Value, and thus it captures dependencies when it runs and instantly returns the result, if the current computed value is still consisntent.

Compute services are registered ~ almost like singletons:

var services = new ServiceCollection();
var fusion = services.AddFusion(); // It's ok to call it many times
// ~ Like service.AddSingleton<[TService, ]TImplementation>()
fusion.AddService<ExampleService>();

Check out CounterService from HelloBlazorServer sample to see the actual code of compute service.

Now, I guess you're curious how the UI code looks like with Fusion You'll be surprised, but it's as simple as it could be:

// MomentsAgoBadge.razor
@inherits ComputedStateComponent<string>
@inject IFusionTime _fusionTime

<span>@State.Value</span>

@code {
    [Parameter] 
    public DateTime Value { get; set; }

    protected override Task<string> ComputeState()
        => _fusionTime.GetMomentsAgo(Value) ;
}

MomentsAgoBadge is Blazor component displays "N [seconds/minutes/...] ago" string. The code above is almost identical to its actual code, which is a bit more complex due to null handling.

You see it uses IFusionTime - one of built-in compute services that provides GetUtcNow and GetMomentsAgo methods. As you might guess,the results of these methods are invalidated automatically; check out FusionTime service to see how it works.

But what's important here is that MomentsAgoBadge is inherited from ComputedStateComponent - an abstract type which provides ComputeState method. As you might guess, this method behaves like a [Compute Method].

ComputedStateComponent<T> exposes State property (of ComputedState<T> type), which allows you to get the most recent output of ComputeState()' via its Value property. "State" is another key Fusion abstraction - it implements a "wait for invalidation and recompute" loop similar to this one:

var computed = await Computed.Capture(_ => service.Method(...));
while (true) {
    await computed.WhenInvalidated();
    computed = await computed.Update();
}

The only difference is that it does this in a more robust way - in particular, it allows you to control the delays between the invalidation and the update, access the most recent non-error value, etc.

Finally, ComputedStateComponent automatically calls StateHasChanged() once its State gets updated to make sure the new value is displayed.

So if you use Fusion, you don't need to code any reactions in the UI. Reactions (i.e. partial updates and re-renders) happen automatically due to dependency chains that connect your UI components with the data providers they use, which in turn are connected to data providers they use, and so on - till the very basic "ingredient providers", i.e. compute methods that are invalidated on changes.

If you want to see a few more examples of similarly simple UI components, check out:

Why Fusion is a game changer for real-time apps?

Real-time typically implies you use events to deliver change notifications to every client which state might be impacted by this change, so you have to:

  1. Know which clients to notify about a particular event. This alone is a fairly hard problem - in particular, you need to know what every client "sees" now. Sending events for anything that's out of the "viewport" (e.g. a post you may see, but don't see right now) doesn't make sense, because it's a huge waste that severely limits the scalability. Similarly to MMORPG, the "visible" part of the state is tiny in comparison to the "available" one for most of web apps too.
  2. Apply events to the client-side state. Kind of an easy problem too, but note that you should do the same on server side as well, and keeping the logic in two completely different handlers in sync for every event is a source of potential problems in future.
  3. Make UI to properly update its event subscriptions on every client-side state change. This is what client-side code has to do to ensure p.1 properly works on server side. And again, this looks like a solvable problem on paper, but things get much more complex if you want to ensure your UI provides a truly eventually consistent view. Just think in which order you'd run "query the initial data" and "subscribe to the subsequent events" actions to see some issues here.
  4. Throttle down the rate of certain events (e.g. "like" events for every popular post). Easy on paper, but more complex if you want to ensure the user sees eventually consistent view on your system. In particular, this implies that every event you send "summarizes" the changes made by it and every event you discard, so likely, you'll need a dedicated type, producer, and handlers for each of such events.

And Fusion solves all these problems using a single abstraction allowing it to identifying and track data dependencies automatically.

Why Fusion is a game changer for Blazor apps with complex UI?

Fusion allows you to create truly independent UI components. You can embed them in any part of UI without any need to worry of how they'll interact with each other.

This makes Fusion a perfect fit for micro-frontends on Blazor: the ability to create loosely coupled UI components is paramount there.

Besides that, if your invalidation logic is correct, Fusion guarantees that your UI state is eventually consistent.

You might think all of this works only in Blazor Server mode. But no, all these UI components work in Blazor WebAssembly mode as well, which is another unique feature Fusion provides. Any Compute Service can be substituted with Compute Service Client, which not simply proxies the calls, but also completely kills the chattiness you'd expect from a regular client-side proxy.

Next Steps

Posts And Other Content

P.S. If you've already spent some time learning about Fusion, please help us to make it better by completing Fusion Feedback Form (1…3 min).