petabridge / NBench

Performance benchmarking and testing framework for .NET applications :chart_with_upwards_trend:
https://nbench.io/
Apache License 2.0
532 stars 46 forks source link

PerfSetup failure can cause NBench.Runner to hang #60

Closed Aaronontheweb closed 8 years ago

Aaronontheweb commented 8 years ago

Have an Akka.NET throughput spec that fails during setup for reasons unknown at the moment:

[PerfSetup]
        public void Setup(BenchmarkContext context)
        {
            _remoteMessageThroughput = context.GetCounter(RemoteMessageCounterName);
            System1 = ActorSystem.Create("SystemA" + Counter.Next(), CreateActorSystemConfig("SystemA" + Counter.Current, "127.0.0.1", 0));
            _echo = System1.ActorOf(Props.Create(() => new EchoActor()), "echo");

            System2 = ActorSystem.Create("SystemB" + Counter.Next(), CreateActorSystemConfig("SystemB" + Counter.Next(), "127.0.0.1", 0));
            _receiver =
                System2.ActorOf(
                    Props.Create(() => new BenchmarkActor(_remoteMessageThroughput, RemoteMessageCount, _resetEvent)),
                    "benchmark");

            var system1Address = RARP.For(System1).Provider.Transport.DefaultAddress;
            var system2Address = RARP.For(System2).Provider.Transport.DefaultAddress;

            var system1EchoActorPath = new RootActorPath(system1Address) / "user" / "echo";
            var system2RemoteActorPath = new RootActorPath(system2Address) / "user" / "benchmark";
            _remoteReceiver =
                System1.ActorSelection(system2RemoteActorPath).Ask<ActorIdentity>(new Identify(null), TimeSpan.FromSeconds(2)).Result.Subject;
            _remoteEcho =
                System2.ActorSelection(system1EchoActorPath)
                    .Ask<ActorIdentity>(new Identify(null), TimeSpan.FromSeconds(2))
                    .Result.Subject;
        }

This code currently throws an exception and causes the NBench.Runner not to proceed onto the next test. I think this is probably an issue inside the Benchmark class itself.

Aaronontheweb commented 8 years ago

No longer true - been able to verify this inside several benchmarks since.