search

realm / realm-dotnet

Realm is a mobile database: a replacement for SQLite & ORMs
https://realm.io
Apache License 2.0
1.24k stars 161 forks source link

Implement optional `[Memoized]` properties in a `RealmObject`/`EmbeddedObject` #2713

Open daawaan4U opened 2 years ago

daawaan4U commented 2 years ago

Description

I'm completely aware this goes against the default zero-copy architecture of Realm, but for the sake of performance critical applications, I do hope this gathers attention.

By default, the properties of a RealmObject are direct pointers to its corresponding data on the disk and do not use any backing fields for caching. This is a boost for the memory because not all the properties of a RealmObject are used for every query anyway. But what if memory is not an issue and certain properties of a certain type inheriting from RealmObject/EmbedddedObject are frequently accessed by the program and what if the program has to individually access hundreds or thousands of these objects?

My proposal is to provide an attribute (e.g. [Memoized]) that will tell Fody to add a backing field for the property and update it whenever the object changes. Below is a simplified example:

// Sample RealmObject
public class Item : RealmObject {
    ...
    public int OrdinaryRealmProperty { get; set; }
    [Memoized] public int MemoizedRealmProperty { get; set; }
    ...
}

// Sample RealmObject after weaving
public class Item : RealmObject {
    ...
    public int OrdinaryRealmProperty { 
        get => ... // default getter method weaved by fody
        set => ... // default setter method weaved by fody
    }

    private int _memoizedRealmProperty;
    public int MemoizedRealmProperty {
        get => _memoizedRealmProperty;
        set => ... // default setter method weaved by fody
    }

    protected override void OnPropertyChanged( string propertyName ) {
        base.OnPropertyChanged( string propertyName );
        _memoizedRealmProperty = ... // default getter method weaved by fody
    }
    ...
}

I strongly believe the zero-copy should still be the default behavior of a property, but I'm looking forward for a similar feature to be implemented for properties that are frequently accessed.

Workaround

One can just simply implement the boilerplate above. Define two properties: one for the persisted property, one for the property that uses a backing field, then override the OnPropertyChanged to always refresh the backing field when the persisted property changes.

For my current use case however, I'm currently trying to attach a set of properties to ALL my RealmObjects so implementing this boilerplate quickly became a mess. Defining an abstract class that inherits RealmObject isn't supported by the weaver either. I can also just use an EmbeddedObject and attach it to all my RealmObjects but I still have to pay the performance penalty of get accessor for the EmbeddedObject property.

The workaround I went with is to just store these values in a Dictionary<TKey, TValue> with the hash code of the RealmObject as keys. Unfortunately, benchmarking showed that the speed gained wasn't large enough (~30-50 nanoseconds) compared to the boilerplate workaround aforementioned (direct access to a backing field, should be almost indistinguishable from an empty method) given how fast already is the woven get accessor for a RealmObject (~150 nanoseconds).

How important is this improvement for you?

Actually, if feasible, I'd even prefer having the option to completely memoize all properties of objects by default in my current case (using Realm for an internal blazor server application) where speed in fetching data is much more of a concern than memory.

nirinchev commented 2 years ago

This is an interesting suggestion. I'll probably need to prepare some perf tests to compare the performance of a memoized property vs what we currently have. Due to the way Realm works, using notifications will incur some perf hit, but I guess it'll be acceptable for read heavy scenarios. Either way, we need to first migrate away from the weaver and start using source generators before we can start looking into this proposal. We're hoping to begin prep work for source generators early next year, so if the performance of the zero-copy getters is not sufficient for your use case, my only suggestion would be to use the workaround you've outlined - it's a good amount of boilerplate, but hopefully it only needs to be written once.

On that note, would you mind sharing some high-level details about the project you're working on? You seem to be pushing Realm to the limits here, which is always interesting. If it's something super secret, that's alright - I'm just curious and we don't need to know the exact use case to evaluate the merits of this proposal 😄

peppy commented 2 years ago

Also very curious about the tight performance requirements here. I did my own benchmarking of property access before we considered migrating to realm (https://github.com/ppy/osu/issues/7057 search for "Property retrival via realm") and noticed reads of property only being around 3x slower than a basic { get; set; } property (as in both were quite low) when managed, and negligible overhead for unmanaged objects. In most real world scenarios this would not be enough to become an issue. I was also seeing no difference based on disk IO, at least with the small test setup I had.

Would be interested to see any benchmarks you have which potentially differ from what I found, as we also have considerably tight performance requirements (in some cases we are accessing realm properties in a synchronous manner from the same thread that is updating our game's UI).

daawaan4U commented 2 years ago

My requirements mainly come from the framework itself (Blazor Server) that I'm using rather than the business details behind the project. Since unlike React or any similar SPA framework, every re-render for every single browser tab opened for every client is performed in the server instead of being executed inside the browser which can be slowed down by Realm's lazy-loading.

Anyway, Realm was meant to run on the client 😅 though it would still be great to see Realm replacing SQLite + ORMs on the server for small server applications especially for realtime cases.

About the benchmarks I made, there's a problem with the property notifications with Realm that I've encountered which I'll mention later. Feel free to examine the code and the results below.

Logs

Before Initialization
Managed - Ordinary Property: False
Managed - Persisted Property: False
Managed - Memoized Property: False
Unmanaged - Ordinary Property: False
Unmanaged - Persisted Property: False
Unmanaged - Memoized Property: False

After Initialization
Managed - Ordinary Property: True
Managed - Persisted Property: True
Managed - Memoized Property: False
Unmanaged - Ordinary Property: True
Unmanaged - Persisted Property: True
Unmanaged - Memoized Property: True

Results

|                     Method |        Mean |     Error |    StdDev |
|--------------------------- |------------:|----------:|----------:|
| Unmanaged_OrdinaryProperty |   5.2505 ns | 0.0308 ns | 0.0288 ns |
|   Managed_OrdinaryProperty | 214.5317 ns | 1.8375 ns | 1.7188 ns |
| Unmanaged_MemoizedProperty |   0.0087 ns | 0.0053 ns | 0.0049 ns |
|   Managed_MemoizedProperty |   0.0033 ns | 0.0034 ns | 0.0030 ns |

Sample Realm Object

public class Item : RealmObject {

    public bool OrdinaryProperty { get; set; }

    public bool PersistedProperty { get; private set; }
    private bool memoizedProperty;
    public bool MemoizedProperty {
        get => memoizedProperty;
        set => PersistedProperty = value;
    }

    protected override void OnPropertyChanged( string propertyName ) {
        if (propertyName == nameof( PersistedProperty )) {
            memoizedProperty = PersistedProperty;
            RaisePropertyChanged( nameof( MemoizedProperty ) );
        }
    }
}

Benchmarks

public class Test_PropertyAccess {

    public static Realm Realm { get; }
    public static Item UnmanagedItem { get; } = new();
    public static Item ManagedItem { get; } = new();

    static Test_PropertyAccess() {

        static void LogProperties( string title ) {
            Console.WriteLine( title );
            Console.WriteLine( $"Managed - Ordinary Property: {ManagedItem.OrdinaryProperty}" );
            Console.WriteLine( $"Managed - Persisted Property: {ManagedItem.PersistedProperty}" );
            Console.WriteLine( $"Managed - Memoized Property: {ManagedItem.MemoizedProperty}" );
            Console.WriteLine( $"Unmanaged - Ordinary Property: {UnmanagedItem.OrdinaryProperty}" );
            Console.WriteLine( $"Unmanaged - Persisted Property: {UnmanagedItem.PersistedProperty}" );
            Console.WriteLine( $"Unmanaged - Memoized Property: {UnmanagedItem.MemoizedProperty}" );
        }

        Realm = Realm.GetInstance( new RealmConfiguration( "Benchmark.realm" ) );

        Realm.Write( () => Realm.Add( ManagedItem ) );
        LogProperties( "Before Initialization" );
        Realm.Write( () => {
            ManagedItem.OrdinaryProperty = true;
            ManagedItem.MemoizedProperty = true;
        } );
        UnmanagedItem.OrdinaryProperty = true;
        UnmanagedItem.MemoizedProperty = true;
        LogProperties( "After Initialization" );
    }

    [Benchmark] public void Unmanaged_OrdinaryProperty() => _ = UnmanagedItem.OrdinaryProperty;
    [Benchmark] public void Managed_OrdinaryProperty() => _ = ManagedItem.OrdinaryProperty;
    [Benchmark] public void Unmanaged_MemoizedProperty() => _ = ManagedItem.MemoizedProperty;
    [Benchmark] public void Managed_MemoizedProperty() => _ = UnmanagedItem.MemoizedProperty;
}

\ I tried comparing it to @peppy's benchmarks and I'm struggling to identify what made the ratio of the results different. One interesting thing though is that there's a difference in accessing a woven property of an unmanaged instance and accessing an ordinary property with a backing field.

Regarding the problem I found, the backing field for the memoized property of the managed instance is somehow not updating in this case yet the ordinary property of the managed instance and the memoized property of the unmanaged instance gets updated. Unfortunately, I wasn't able to identify the cause and wasn't able to make a workaround for this.

nirinchev commented 2 years ago

Accessing a "regular" Realm property, even for an unmanaged object, is inevitably going to be slower, because the woven property introduces an if-check - it looks like:

public bool OrdinaryProperty
{
    get
    {
        if (IsManaged)
        {
            return GetValue("OrdinaryProperty");
        }

        return backingField;
    }
}

Still, I'm kind of surprised this results in a 1000x difference in performance, especially considering Benchmark.Net does warmup invocations, which should be sufficient to coach the branch prediction algorithms to select the correct branch during the actual measured tests.

Regarding your issue - Realm will not invoke OnPropertyChanged unless you have subscribed for notifications. This is because calculating notifications is not free, so we're not doing it unless there's someone actually listening for those.

daawaan4U commented 2 years ago

I tried subscribing to the managed instance and it still won't show any signs of notifying it though. The modified code for the benchmark is provided below. Sad to say, no exceptions were thrown even for the OrdinaryProperty

public class Test_PropertyAccess {
    ...
    static Test_PropertyAccess() {
        ...
        LogProperties( "Before Initialization" );
        ManagedItem.PropertyChanged += ( s, e ) => throw new Exception(); // I only inserted this
        Realm.Write( () => {
        ...
    }
    ...
}
daawaan4U commented 2 years ago

Just a short update, I haven't fixed the memoized property yet for the managed instance not being updated. On a sidenote, I rewrote the benchmarks and tested it with multiple objects instead since testing one object only seemed unreliable. I noticed that the difference between accessing an ordinary property of an unmanaged instance and a memoized property of a managed instance became negligible at a large number of objects. I'll reply back for the results tomorrow. I'll be leaving the computer overnight to run this, here's the code:

Revised Sample Realm Object

public class Item : RealmObject {

    public int OrdinaryProperty { get; set; }

    public int PersistedProperty { get; private set; }
    public int memoizedPropertyField;
    public int MemoizedProperty {
        get => memoizedPropertyField;

        // Directly assign to backing field for now.
        // In production, backing field should be updated by notifiers instead
        set => PersistedProperty = memoizedPropertyField = value;
    }

    protected override void OnPropertyChanged( string propertyName ) {
        if (propertyName == nameof( PersistedProperty )) {
            memoizedPropertyField = PersistedProperty;
            RaisePropertyChanged( nameof( MemoizedProperty ) );
        }
    }
}

Revised Benchmark

[CsvExporter]
[GroupBenchmarksBy( BenchmarkLogicalGroupRule.ByCategory )]
public class Test_PropertyAccess {

    [Params( 1, 5, 10, 25, 50, 100, 250, 500, 1_000, 2_500, 5_000, 10_000, 25_000, 50_000, 100_000 )]
    public int Count { get; set; }

    public static RealmConfiguration RealmConfiguration { get; } = new RealmConfiguration( "Benchmark.realm" );
    public static Item[] UnmanagedItems { get; set; } = new Item[ 1_000_000 ];
    public static Item[] ManagedItems { get; set; } = new Item[ 1_000_000 ];

    static Test_PropertyAccess() {
        Realm.DeleteRealm( RealmConfiguration );
        var realm = Realm.GetInstance( RealmConfiguration );

        for (int index = 0; index < 1_000_000; index++) {
            var num = Random.Shared.Next();
            UnmanagedItems[ index ] = new Item { MemoizedProperty = num, OrdinaryProperty = num };
            ManagedItems[ index ] = new Item { MemoizedProperty = num, OrdinaryProperty = num };
        }

        realm.Write( () => realm.Add( ManagedItems ) );
    }

    #region
    // Should I even include the following two? I'm relatively new to benchmarking, my apologies
    // I honestly do not know if these are still needed
    [BenchmarkCategory( "Unmanaged" ), Benchmark]
    public void Unmanaged_LoopAccessOverhead() {
        for (int index = 0; index < Count; index++)
            _ = UnmanagedItems[ index ];
    }

    [BenchmarkCategory( "Managed" ), Benchmark]
    public void Managed_LoopAccessOverhead() {
        for (int index = 0; index < Count; index++)
            _ = ManagedItems[ index ];
    }
    #endregion

    [BenchmarkCategory( "Unmanaged" ), Benchmark]
    public void Unmanaged_OrdinaryProperty() {
        for (int index = 0; index < Count; index++)
            _ = UnmanagedItems[ index ].OrdinaryProperty;
    }

    [BenchmarkCategory( "Managed" ), Benchmark]
    public void Managed_OrdinaryProperty() {
        for (int index = 0; index < Count; index++)
            _ = ManagedItems[ index ].OrdinaryProperty;
    }

    [BenchmarkCategory( "Unmanaged" ), Benchmark( Baseline = true )]
    public void Unmanaged_MemoizedProperty() {
        for (int index = 0; index < Count; index++)
            _ = UnmanagedItems[ index ].MemoizedProperty;
    }

    [BenchmarkCategory( "Managed" ), Benchmark( Baseline = true )]
    public void Managed_MemoizedProperty() {
        for (int index = 0; index < Count; index++)
            _ = ManagedItems[ index ].MemoizedProperty;
    }
}
daawaan4U commented 2 years ago

Here are the results. I also noticed that the difference between accessing the ordinary property of a managed instanced and the memoized property eventually became smaller at larger number of objects:

|                       Method |  Count |               Mean |           Error |          StdDev |             Median |    Ratio | RatioSD |
|----------------------------- |------- |-------------------:|----------------:|----------------:|-------------------:|---------:|--------:|
| Unmanaged_LoopAccessOverhead |      1 |          0.4633 ns |       0.0116 ns |       0.0103 ns |          0.4648 ns |     3.16 |    0.22 |
|   Unmanaged_OrdinaryProperty |      1 |          5.1697 ns |       0.0247 ns |       0.0231 ns |          5.1637 ns |    35.50 |    2.38 |
|   Unmanaged_MemoizedProperty |      1 |          0.1463 ns |       0.0109 ns |       0.0102 ns |          0.1448 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |      1 |          0.3766 ns |       0.0113 ns |       0.0100 ns |          0.3777 ns |     2.34 |    0.08 |
|     Managed_OrdinaryProperty |      1 |        346.3105 ns |       1.3770 ns |       1.2207 ns |        346.3225 ns | 2,153.36 |   58.80 |
|     Managed_MemoizedProperty |      1 |          0.1609 ns |       0.0049 ns |       0.0043 ns |          0.1614 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |      5 |          2.4317 ns |       0.0425 ns |       0.0398 ns |          2.4275 ns |     0.86 |    0.01 |
|   Unmanaged_OrdinaryProperty |      5 |          7.3722 ns |       0.0308 ns |       0.0257 ns |          7.3801 ns |     2.60 |    0.01 |
|   Unmanaged_MemoizedProperty |      5 |          2.8301 ns |       0.0184 ns |       0.0154 ns |          2.8268 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |      5 |          2.2243 ns |       0.0718 ns |       0.0637 ns |          2.2228 ns |     0.79 |    0.03 |
|     Managed_OrdinaryProperty |      5 |      1,738.4931 ns |      11.5426 ns |      10.2322 ns |      1,737.1017 ns |   614.29 |    7.28 |
|     Managed_MemoizedProperty |      5 |          2.8304 ns |       0.0378 ns |       0.0335 ns |          2.8309 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |     10 |          4.6553 ns |       0.0342 ns |       0.0303 ns |          4.6611 ns |     0.71 |    0.01 |
|   Unmanaged_OrdinaryProperty |     10 |         10.8958 ns |       0.0775 ns |       0.0687 ns |         10.8907 ns |     1.65 |    0.02 |
|   Unmanaged_MemoizedProperty |     10 |          6.5869 ns |       0.0582 ns |       0.0545 ns |          6.5648 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |     10 |          4.6489 ns |       0.0449 ns |       0.0420 ns |          4.6305 ns |     0.71 |    0.01 |
|     Managed_OrdinaryProperty |     10 |      3,460.2167 ns |      25.5389 ns |      22.6396 ns |      3,461.3005 ns |   524.81 |    6.25 |
|     Managed_MemoizedProperty |     10 |          6.5909 ns |       0.0733 ns |       0.0686 ns |          6.5744 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |     25 |         17.6830 ns |       0.1175 ns |       0.1041 ns |         17.6975 ns |     0.84 |    0.01 |
|   Unmanaged_OrdinaryProperty |     25 |         27.1708 ns |       0.0847 ns |       0.0793 ns |         27.1483 ns |     1.28 |    0.01 |
|   Unmanaged_MemoizedProperty |     25 |         21.1673 ns |       0.1622 ns |       0.1438 ns |         21.1564 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |     25 |         17.2132 ns |       0.1285 ns |       0.1202 ns |         17.2276 ns |     0.81 |    0.01 |
|     Managed_OrdinaryProperty |     25 |      8,309.8856 ns |      46.1663 ns |      40.9252 ns |      8,303.8506 ns |   390.51 |    2.81 |
|     Managed_MemoizedProperty |     25 |         21.2956 ns |       0.1912 ns |       0.1789 ns |         21.3112 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |     50 |         27.5601 ns |       0.1587 ns |       0.1407 ns |         27.5378 ns |     0.83 |    0.01 |
|   Unmanaged_OrdinaryProperty |     50 |         41.8492 ns |       0.2018 ns |       0.1888 ns |         41.8434 ns |     1.26 |    0.01 |
|   Unmanaged_MemoizedProperty |     50 |         33.1537 ns |       0.1607 ns |       0.1424 ns |         33.1251 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |     50 |         27.5440 ns |       0.1035 ns |       0.0917 ns |         27.5518 ns |     0.88 |    0.01 |
|     Managed_OrdinaryProperty |     50 |     16,666.8578 ns |     204.2315 ns |     191.0383 ns |     16,663.4079 ns |   533.84 |    6.25 |
|     Managed_MemoizedProperty |     50 |         31.1440 ns |       0.2579 ns |       0.2154 ns |         31.1487 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |    100 |         47.2583 ns |       0.0964 ns |       0.0753 ns |         47.2671 ns |     0.83 |    0.00 |
|   Unmanaged_OrdinaryProperty |    100 |         71.0297 ns |       0.4219 ns |       0.3946 ns |         71.0707 ns |     1.25 |    0.01 |
|   Unmanaged_MemoizedProperty |    100 |         57.0116 ns |       0.2140 ns |       0.1897 ns |         57.0222 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |    100 |         40.8856 ns |       0.2252 ns |       0.1996 ns |         40.9477 ns |     0.74 |    0.00 |
|     Managed_OrdinaryProperty |    100 |     34,018.4143 ns |     278.0266 ns |     232.1648 ns |     34,108.4900 ns |   617.59 |    4.96 |
|     Managed_MemoizedProperty |    100 |         55.1043 ns |       0.2999 ns |       0.2658 ns |         55.0692 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |    250 |        105.6196 ns |       0.4351 ns |       0.3633 ns |        105.7251 ns |     0.84 |    0.01 |
|   Unmanaged_OrdinaryProperty |    250 |        159.3590 ns |       0.5185 ns |       0.4850 ns |        159.4721 ns |     1.26 |    0.01 |
|   Unmanaged_MemoizedProperty |    250 |        126.3496 ns |       0.7727 ns |       0.6850 ns |        126.4173 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |    250 |        105.4486 ns |       0.4923 ns |       0.4605 ns |        105.5688 ns |     0.82 |    0.00 |
|     Managed_OrdinaryProperty |    250 |     85,362.8967 ns |     632.9363 ns |     561.0816 ns |     85,407.1228 ns |   665.89 |    3.43 |
|     Managed_MemoizedProperty |    250 |        128.1952 ns |       0.7607 ns |       0.6744 ns |        128.2239 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |    500 |        202.0967 ns |       1.3705 ns |       1.2149 ns |        201.8735 ns |     0.65 |    0.01 |
|   Unmanaged_OrdinaryProperty |    500 |        387.6735 ns |       1.4647 ns |       1.2984 ns |        387.7765 ns |     1.25 |    0.01 |
|   Unmanaged_MemoizedProperty |    500 |        309.7384 ns |       1.8640 ns |       1.5566 ns |        309.3799 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |    500 |        202.9362 ns |       1.5313 ns |       1.4324 ns |        202.6465 ns |     0.62 |    0.01 |
|     Managed_OrdinaryProperty |    500 |    170,466.7097 ns |     429.8715 ns |     381.0699 ns |    170,556.5674 ns |   518.10 |    2.15 |
|     Managed_MemoizedProperty |    500 |        329.0289 ns |       1.4049 ns |       1.2454 ns |        328.8160 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |   1000 |        320.8106 ns |       1.8301 ns |       1.7118 ns |        321.1231 ns |     0.44 |    0.00 |
|   Unmanaged_OrdinaryProperty |   1000 |        802.2971 ns |       5.9716 ns |       5.2937 ns |        800.1520 ns |     1.11 |    0.01 |
|   Unmanaged_MemoizedProperty |   1000 |        722.2206 ns |       3.5318 ns |       3.1309 ns |        722.5036 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |   1000 |        397.6760 ns |       2.2702 ns |       2.1236 ns |        397.2885 ns |     0.55 |    0.00 |
|     Managed_OrdinaryProperty |   1000 |    338,080.5804 ns |   3,320.9384 ns |   2,943.9254 ns |    338,568.5303 ns |   464.72 |    4.79 |
|     Managed_MemoizedProperty |   1000 |        727.5169 ns |       3.7080 ns |       3.2870 ns |        727.8455 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |   2500 |        979.8451 ns |       4.3595 ns |       3.8646 ns |        979.3230 ns |     0.37 |    0.00 |
|   Unmanaged_OrdinaryProperty |   2500 |      2,884.9155 ns |      24.1393 ns |      26.8307 ns |      2,880.2582 ns |     1.11 |    0.01 |
|   Unmanaged_MemoizedProperty |   2500 |      2,613.3421 ns |      17.0380 ns |      14.2275 ns |      2,612.9433 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |   2500 |        980.4956 ns |       3.7423 ns |       3.3174 ns |        980.2743 ns |     0.38 |    0.00 |
|     Managed_OrdinaryProperty |   2500 |    839,346.6276 ns |   6,234.3690 ns |   5,831.6327 ns |    838,740.2344 ns |   322.41 |    2.94 |
|     Managed_MemoizedProperty |   2500 |      2,600.5882 ns |      16.3634 ns |      13.6642 ns |      2,602.5078 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |   5000 |      1,947.3948 ns |      12.8111 ns |      11.3567 ns |      1,948.0333 ns |     0.30 |    0.00 |
|   Unmanaged_OrdinaryProperty |   5000 |      7,791.6220 ns |     311.6131 ns |     918.7981 ns |      7,272.4442 ns |     1.17 |    0.14 |
|   Unmanaged_MemoizedProperty |   5000 |      6,585.7422 ns |      53.0760 ns |      47.0505 ns |      6,578.1025 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |   5000 |      1,953.8181 ns |      10.4549 ns |       9.7795 ns |      1,954.7382 ns |     0.30 |    0.00 |
|     Managed_OrdinaryProperty |   5000 |  1,667,407.6172 ns |  11,076.2648 ns |   8,647.6190 ns |  1,669,890.8203 ns |   252.14 |    1.32 |
|     Managed_MemoizedProperty |   5000 |      6,608.4450 ns |      27.7789 ns |      24.6253 ns |      6,616.9617 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |  10000 |      3,906.6359 ns |      19.8554 ns |      17.6013 ns |      3,910.1185 ns |     0.27 |    0.02 |
|   Unmanaged_OrdinaryProperty |  10000 |     15,340.3807 ns |     556.7125 ns |   1,632.7415 ns |     14,388.0051 ns |     1.07 |    0.14 |
|   Unmanaged_MemoizedProperty |  10000 |     14,515.0930 ns |     527.6101 ns |   1,555.6702 ns |     13,635.5453 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |  10000 |      3,926.3088 ns |      37.7796 ns |      33.4906 ns |      3,915.2718 ns |     0.27 |    0.03 |
|     Managed_OrdinaryProperty |  10000 |  3,296,655.0781 ns |  22,714.6888 ns |  18,967.7939 ns |  3,299,379.2969 ns |   222.59 |   27.07 |
|     Managed_MemoizedProperty |  10000 |     14,211.5454 ns |     518.0174 ns |   1,527.3861 ns |     13,252.7878 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |  25000 |      9,737.3346 ns |      71.5333 ns |      63.4124 ns |      9,733.2558 ns |     0.29 |    0.00 |
|   Unmanaged_OrdinaryProperty |  25000 |     35,548.4548 ns |     697.5262 ns |     652.4664 ns |     35,354.9103 ns |     1.07 |    0.02 |
|   Unmanaged_MemoizedProperty |  25000 |     33,147.0586 ns |     644.5296 ns |     503.2063 ns |     32,982.7454 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |  25000 |      9,695.4182 ns |      44.8318 ns |      37.4366 ns |      9,694.0758 ns |     0.29 |    0.00 |
|     Managed_OrdinaryProperty |  25000 |  9,054,918.5268 ns |  99,463.6476 ns |  88,171.9329 ns |  9,082,664.0625 ns |   273.86 |    3.02 |
|     Managed_MemoizedProperty |  25000 |     33,080.9708 ns |     341.2932 ns |     266.4593 ns |     33,061.6333 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead |  50000 |     15,568.2755 ns |      74.0747 ns |      65.6653 ns |     15,570.9076 ns |     0.17 |    0.01 |
|   Unmanaged_OrdinaryProperty |  50000 |     99,084.8301 ns |   1,960.9874 ns |   3,053.0212 ns |     97,886.3098 ns |     1.08 |    0.05 |
|   Unmanaged_MemoizedProperty |  50000 |     91,883.7759 ns |   1,820.4725 ns |   3,188.4107 ns |     90,705.2124 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead |  50000 |     19,496.2253 ns |     169.2617 ns |     158.3275 ns |     19,453.5538 ns |     0.21 |    0.01 |
|     Managed_OrdinaryProperty |  50000 | 18,630,875.0000 ns |  98,610.8938 ns |  87,415.9889 ns | 18,624,834.3750 ns |   201.04 |    5.68 |
|     Managed_MemoizedProperty |  50000 |     92,964.8985 ns |   1,811.5833 ns |   2,655.3953 ns |     92,127.1729 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
| Unmanaged_LoopAccessOverhead | 100000 |     31,091.0623 ns |     222.7636 ns |     197.4742 ns |     31,114.1602 ns |     0.07 |    0.00 |
|   Unmanaged_OrdinaryProperty | 100000 |    453,869.5696 ns |   2,951.0659 ns |   2,616.0431 ns |    454,139.5752 ns |     1.09 |    0.01 |
|   Unmanaged_MemoizedProperty | 100000 |    417,885.8154 ns |   2,312.1405 ns |   2,049.6524 ns |    418,421.4844 ns |     1.00 |    0.00 |
|                              |        |                    |                 |                 |                    |          |         |
|   Managed_LoopAccessOverhead | 100000 |     38,849.3953 ns |     142.9941 ns |     126.7606 ns |     38,856.8237 ns |     0.09 |    0.00 |
|     Managed_OrdinaryProperty | 100000 | 36,382,624.2857 ns | 274,291.1713 ns | 256,572.1318 ns | 36,343,657.1429 ns |    87.37 |    0.84 |
|     Managed_MemoizedProperty | 100000 |    416,437.2070 ns |   2,195.0586 ns |   2,053.2592 ns |    416,401.5625 ns |     1.00 |    0.00 |
peppy commented 2 years ago

Thanks for the benchmark code! I actually found an oversight in my original benchmarks when comparing (the "unmanaged" objects I was benchmarking were actually being managed).

Your results look correct (and I get matching results when running locally), although as you say the #regioned benchmarks are probably unnecessary.

daawaan4U commented 2 years ago

Thanks for the response! I wanted to ask, any insights as to why the ratio between the duration of accessing an ordinary property and a memoized property is iconsistent as the number of objects grow? I wasn't able to identify the cause

peppy commented 2 years ago

Very hard to say without looking into detail, but do consider that you're looking at nanosecond values here. It seems to stabilise at all your tests above 1-10 count runs and the ratio is not always moving in one direction either. It could be anything from fluctuating system performance to PGO optimisations.

Even with the two levels of magnitude difference in property retrievals, I'd probably look to be benchmarking in a more real-world usage scenario. For us at least, accesses to properties are quite rare (at the point of display, usually) so the incurred overhead is very low. One exception we have (I haven't finished migrating to realm yet so this is yet to be resolved/revisited) is doing regex searching over a large number of properties, which could be a bit of an issue.

But based on your results, 5,000 property accesses are only around 1.5ms, which in terms of rendering a web page is likely a non-issue (and also likely going to be faster than sqlite) for most scenarios. Definitely slower than no realm backing, but also seems memoizing every property may be premature optimisation. You know your scenario better than I can though :)

carbonete commented 2 years ago

Have my vote 💯 I have a reader app running at windows and android. I have suggestion , be more interesting and maybe "easy path" for some scenery use of InMemoryConfiguration("some-identifier"); open a existing local Realm in read-only mode. Can create a realm to store some data that is used to validate or describe codes, like genres, postal code, relationship and others. This make sense ?

thanks

thygrrr commented 1 month ago

We're working on just memoizing the values we care for ourselves. It requires discarding them on every property change. It's a small amount of additional code and could be fully automated somewhat easily (e.g. via an attribute) if the need arises.