Poor performance when compared to precompiled code

[pavlexander]

Hi

I am currently looking for a solution that would allow users to write the function that would filter out some of the "results". The function has a bool return type and it should take a few input parameters that could be used in filtering logic, i.e. native .Net filter function would look like this:

        public bool Filter(MyItem item, MyStats stats)
        {
            var res1 = item.Sth;
            var res2 = stats.SomeResults;

            if (res2[0] >= 0)
            {
                return true;
            }

            return false;
        }

Now I am trying to execute the same function using the Westwind.Scripting, to imitate the user-created filter function:

            var stats = new MyStats() { SomeResults = new decimal[] { 0, 15, 30, 45, 100 }};

            var items = new List<MyItem>();
            for (int i = 0; i < 10_000_000; i++)
                items.Add(new MyItem() { Sth = i });

            var exec = new CSharpScriptExecution() { SaveGeneratedCode = true };
            exec.AddDefaultReferencesAndNamespaces();

            exec.AddAssembly(typeof(MyItem));
            exec.AddAssembly(typeof(MyStats));
            exec.AddNamespace("ClassLibrary1");

            var code = @"
public bool Filter(MyItem item, MyStats stats)
{
var res1 = item.Sth;
var res2 = stats.SomeResults;

if (res2[0] >= 0)
{
    return true;
}

return false;
}
";
            int numberOfPositives = 0;
            var sw = Stopwatch.StartNew();
            for (int i = 0; i < items.Count(); i++)
            {
                var item = items[i];

                bool result = exec.ExecuteMethod<bool>(code, "Filter", item, stats); // 10sec+
                //var result = Filter(item, stats); // sub 1 sec

                if (res)
                {
                    numberOfPositives++;
                }
            }

            sw.Stop();
            MessageBox.Show($"Total elapsed milliseconds: {sw.ElapsedMilliseconds}");

and the classes are:

namespace ClassLibrary1
{
    public class MyStats
    {
        public decimal[] SomeResults { get; set; }
    }

    public class MyItem
    {
        public int Sth { get; set; }
    }
}

The problem is

• when I execute the code natively - it runs blazingly fast 249ms
• when I execute the script with runtime compilation (Westwind.Scripting) the execution time is 11_141ms

I was under the impression that the performance would be nearly the same as for the precompiled code. Since the assembly is getting compiled into in-memory dll once, and then get's re-executed just as a standard assembly would?

is there a way to speed up the solution?

note: I apologize for classes and names that do not make sense :) I am just prototyping the solution and trying to make a minimum viable working version of it without paying to much attention to names and "logic" etc. Per use-case the filtering function will be called tens thousands time in a loop, so the code is still perfectly valid from that perspective.

[pavlexander]

Edit: I am using .Net 6, WinForms app. Library version is the latest 1.2.1

[RickStrahl]

If you are repeatedly running code and performance is really important (ie. you're running something small that is time critical) you should grab an instance of the class and then use Reflection directly to call it. Use CompileClass() to get the reference, then use Reflection or dynamic directly, or the Reflection helpers on the scripting library.

Calling ExecuteCode() repeatedly is going to be much slower even though it doesn't recompile the code each time, it has to check for the code existence find the assembly, load the type each time using Reflection which is going to slow things down a lot. 11s seems very slow though so something else is probably wrong. Make sure you're testing in Release, not with debug too.

All that said dynamic code will never have the same perf as native direct binding so even with CompileClass() and direct Relfection execution you're probably going to be 10x slower than direct binding method invocation.

[pavlexander]

@RickStrahl

I have tried running the previous solution in release mode as well and the execution time was 8 sec, down from 11 so I did not mention it because it was still bad :)

Thank you! this is perfect! Now it only takes 316ms to execute the loop.

            var exec = new CSharpScriptExecution() { SaveGeneratedCode = true };
            exec.AddDefaultReferencesAndNamespaces();

            exec.AddAssembly(typeof(MyItem));
            exec.AddAssembly(typeof(MyStats));

            var code = $@"
using System;
using System.Collections.Generic;
using ClassLibrary1;

namespace MyApp
{{
    public class MyFilter
    {{
                public bool Filter(MyItem item, MyStats stats)
                {{
                        var res1 = item.Sth;
                        var res2 = stats.SomeResults;

                        if (res2[0] >= 0)
                        {{
                            return true;
                        }}

                        return false;
                }}
    }}
}}";

            dynamic math = exec.CompileClass(code);
            bool res = math.Filter(item, stats);

ALSO, while I was waiting for your answer I was playing around with other solutions, and noticed that there is a substantial difference in execution time when using the delegates, as opposed to direct method invoke (or the dynamic invoke). I.e. consider following modification:

            dynamic math = exec.CompileClass(code);

            Type t = math.GetType();
            var methodInfo = t.GetMethod("Filter", new Type[] { typeof(MyItem), typeof(MyStats) });
            if (methodInfo == null) // the method doesn't exist
            {
                throw new Exception();
            }

            var func = (Func<MyItem, MyStats, bool>)methodInfo.CreateDelegate(typeof(Func<MyItem, MyStats, bool>), math);
            bool res = func(item, stats);

With this the run time is somewhere between 180 and 240 ms. In fact, this approach works even faster than the raw/direct method call bool res = Filter(item, stats); ~280ms.. Do you maybe have an explanation of why that is?

Could you tell me if I have to manage the assembly unload somehow? In .net Framework we could load assemblies in separate AppDomain and in .net core+ we shall use the assembly load context.

Does your library utilize any of the these approaches or does it load the assembly into the main app domain context? If possible I would like to achieve some degree if isolation..

[RickStrahl]

Sure - when you create a delegate you're pinning the types down and aren't using dynamic lookups/Reflection. Dynamic and Reflection are always slower, but they are considerably easier.

I'm not sure what your loop looks like but for most situations a 100ms over a 10,000 iteration loop is not worth fussing over unless it's a specific scenario that microsecond sensitive in which case you probably shouldn't be using script code in the first place. :smile:

[jasonswearingen]

This is really interesting information (regarding performance). @RickStrahl could you make a wiki or doc page with some of these tips?

[pavlexander]

Thank you for help! I have no further questions. Should I close the topic or do you use it for tracking of wiki update? Please let me know if I should close it.

Thank you very much for help.

[RickStrahl]

I've added a small section to the FAQ section of the README file:

FAQ - Performance