Exception try catch performance 10 times low than java

[chrishaly]

A compare test did, result says: performance of try catch throw Exception in .Net is over 10 times slow than in Java.

Reproduce:

C# code

    class ExceptionPerformanceTest
    {
        public void Test()
        {
            var stopwatch = Stopwatch.StartNew();
            ExceptionTest(100_000);
            stopwatch.Stop();
            Console.WriteLine(stopwatch.ElapsedMilliseconds);
        }

        private void ExceptionTest(long times)
        {
            for (int i = 0; i < times; i++)
            {
                try
                {
                    throw new Exception();
                }
                catch (Exception ex)
                {
                    //Ignore
                }
            }
        }
    }

Java Code

public class ExceptionPerformanceTest {

    public void Test() {
        Instant start = Instant.now();
        ExceptionTest(100_000);
        Instant end = Instant.now();
        Duration duration = Duration.between(start, end);
        System.out.println(duration.toMillis());

    }

    private void ExceptionTest(long times) {
        for (int i = 0; i < times; i++) {
            try {
                throw new Exception();
            } catch (Exception ex) {
                //Ignore
            }
        }
    }
}

Test Result:

.Net: 2151ms Java: 175ms

2151/175=12.29

source code at https://github.com/chrishaly/DotNetVsJavaPerformanceTest

[Symbai]

For me it seems like there is a huge regression on .NET Core comparing to .NET Framework.

BenchmarkDotNet=v0.11.5, OS=Windows 10.0.18362
Intel Core i7-4960X CPU 3.60GHz (Haswell), 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=3.0.100-preview4-011223
  [Host]     : .NET Core 3.0.0-preview4-27615-11 (CoreCLR 4.6.27615.73, CoreFX 4.700.19.21213), 64bit RyuJIT
  Job-HBDUYZ : .NET Core 2.2.3 (CoreCLR 4.6.27414.05, CoreFX 4.6.27414.05), 64bit RyuJIT
  Clr        : .NET Framework 4.7.2 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.8.3801.0
  Core       : .NET Core 3.0.0-preview4-27615-11 (CoreCLR 4.6.27615.73, CoreFX 4.700.19.21213), 64bit RyuJIT

Method	Job	Runtime	Toolchain	Mean	Error	StdDev	Rank	Gen 0	Gen 1	Gen 2	Allocated
Run	Default	Core	.NET Core 2.2	2,080.9 ms	49.455 ms	46.260 ms	2	3000.0000	-	-	20.6 MB
Run	Clr	Clr	Default	685.3 ms	3.420 ms	3.031 ms	1	4000.0000	-	-	27.55 MB
Run	Core	Core	Default	2,190.4 ms	43.222 ms	36.092 ms	3	3000.0000	-	-	19.07 MB

Code:

[ClrJob]
    [CoreJob]
    [MemoryDiagnoser]
    [RPlotExporter, RankColumn]
    public class ExceptionTest
    {
        [Benchmark]
        public bool Run()
        {
            for (int i = 0; i < 100_000L; i++)
            {
                try
                {
                    throw new Exception();
                }
                catch (Exception ex)
                {
                    //Ignore
                }
            }
            return true;
        }     
    }

    class Program
    {
        static void Main(string[] args)
        {
            var summary = BenchmarkRunner.Run<ExceptionTest>(DefaultConfig.Instance.With(Job.Default.With(CsProjCoreToolchain.NetCoreApp22)));
        }
    }

[stephentoub]

For me it seems like there is a huge regression on .NET Core comparing to .NET Framework.

See https://github.com/dotnet/coreclr/issues/22224#issuecomment-499657814

[chrishaly]

Tested on Virtual Machine Ubuntu 18.04.2 LTS

.NetCore 2.2: 3405 Java JDK 11: 237

3405/237=14.367

[mattwarren]

Benchmarking in Java is hard to get right, more so than .NET. You should consider writing a JMH Benchmark to ensure that the timings are correct (BenchmarkDotNet was partly inspired by JMH, so the benchmark code will look similar).

[chrishaly]

Benchmarking in Java is hard to get right, more so than .NET. You should consider writing a JMH Benchmark to ensure that the timings are correct (BenchmarkDotNet was partly inspired by JMH, so the benchmark code will look similar).

Times changed from 100,000 to 1000,000 on windows the result is:

.NetCore 2.2: 30401ms Java JDK 1.8: 726ms

30401ms/726=41.87

So mybe it's not need to get accurate time, since more than 40 times 9.9 or 10.1, 39 or 41 are similar. It's all said that .net core 2.2 exeception handling more slow than java.

To avoid java compiler optimize the code, a counter to sum exception times added. and the result is:

1000000 672

1000000 times exception throw and catched in fact.

    public void Test() {
        Instant start = Instant.now();
        System.out.println(ExceptionTest(1000_000));
        Instant end = Instant.now();
        Duration duration = Duration.between(start, end);
        System.out.println(duration.toMillis());

    }

    private long ExceptionTest(long times) {
        long result = 0;
        for (int i = 0; i < times; i++) {
            try {
                throw new Exception();
            } catch (Exception ex) {
                //Ignore
                result++;
            }
        }
        return result;
    }

[Suchiman]

Just upping a counter doesn't mean it doesn't get optimized away, you would need to throw from a different method and block that method from being inlined, except that i couldn't find a hint to block the JVM from inlining a method

public void Test() {
    Instant start = Instant.now();
    System.out.println(ExceptionTest(1000_000));
    Instant end = Instant.now();
    Duration duration = Duration.between(start, end);
    System.out.println(duration.toMillis());

}

private long ExceptionTest(long times) {
    long result = 0;
    for (int i = 0; i < times; i++) {
        try {
            Throw();
        } catch (Exception ex) {
        }
    }
    return result;
}

//@NoInlining <-- would need something like this
private void Throw() throws Exception {
    throw new Exception();
}

[GSPP]

Exception performance in .NET is known to be slow. I think this is becoming more and more of a problem because:

• Callstacks with await rethrow an exception many times thereby multiplying the costs to an extent.
• Modern applications tend to do more IO. Everything is networked these days (microservices, cloud services). IO is a major source of exceptions.

I have worked on multiple applications where exceptions became a performance issue. The code was not bad. Fairly frequent errors were in the nature of the application.

Also, when the Visual Studio Debugger is attached exceptions again become 10-100x slower.

[chrishaly]

@mattwarren exception handling slow is known, however compare to java .net exception handling 10 or 40 times slow is surprise, two platform both with GC different so much, so there must be some problem in .NET

[janvorli]

I've tried to profile the C# EH example above on Windows. 24% of the time is spent in RaiseException, 17% in RtlLookupFunctionEntry, 10% in RtlVirtualUnwind, 6.4% in EtwEventWrite, 5.5% in EtwEventWriteTransfer and 2.7% in RtlUnwindEx. All the percentages are exclusive. It sums up to 65.6%. All of these functions are in the Windows ntdll.dll, so .NET Core cannot do much about this time. The remaining 35% of the time is split into tiny fragments of 0..2.5% spent in various coreclr functions.

[jkotas]

.NET Core cannot do much about this time

.NET Core can avoid this overhead by handling the exceptions internally, without involving the Windows OS exception handling. It is what CoreRT does. CoreRT runs the exception handling microbenchmarks like the one above number of times faster. There is nothing fundamental (except the huge amount of work involved) that prevents porting the same scheme to CoreCLR.

[davidfowl]

It might be good to tackle this in .NET 6. I saw some of these stacks when I was debugging a performance run. It seems like there's some lock in the OS when exceptions get thrown. There were lots of concurrent sockets disconnecting at the same time and it resulting in high CPU.

[wasabii]

I'm the current maintainer of IKVM. I can vouch for this. :)