Unable to get gateways from Kube objects for cluster 'Test'

[joepvand]

I deployed my project using the instructions in the 'sample' directory, but i keep getting the following error:

fail: Orleans.Clustering.Kubernetes.KubeGatewayListProvider[0] Unable to get gateways from Kube objects for cluster Test System.Net.Http.HttpRequestException: Connection timed out (10.96.0.1:443) ---> System.Net.Sockets.SocketException (110): Connection timed out at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|277_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at k8s.Kubernetes.<>c.<<CreateHttpClient>b__820_0>d.MoveNext() --- End of stack trace from previous location --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp2ConnectionAsync(HttpRequestMessage request) at System.Threading.Tasks.TaskCompletionSourceWithCancellation1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttp2ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpClient.gCore|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at k8s.Kubernetes.ListNamespacedCustomObjectWithHttpMessagesAsync(String group, String version, String namespaceParameter, String plural, Nullable1 allowWatchBookmarks, String continueParameter, String fieldSelector, String labelSelector, Nullable1 limit, String resourceVersion, String resourceVersionMatch, Nullable1 timeoutSeconds, Nullable1 watch, Nullable1 pretty, Dictionary2 customHeaders, CancellationToken cancellationToken) at k8s.KubernetesExtensions.ListNamespacedCustomObjectAsync(IKubernetes operations, String group, String version, String namespaceParameter, String plural, Nullable1 allowWatchBookmarks, String continueParameter, String fieldSelector, String labelSelector, Nullable1 limit, String resourceVersion, String resourceVersionMatch, Nullable1 timeoutSeconds, Nullable1 watch, Nullable`1 pretty, CancellationToken cancellationToken) at Orleans.Clustering.Kubernetes.KubeGatewayListProvider.GetGateways() fail: Orleans.Messaging.GatewayManager[100921] Exception occurred during RefreshSnapshotLiveGateways_TimerCallback -> listProvider.GetGateways() System.Net.Http.HttpRequestException: Connection timed out (10.96.0.1:443) ---> System.Net.Sockets.SocketException (110): Connection timed out at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.gWaitForConnectWithCancellation|277_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at k8s.Kubernetes.<>c.<b__820_0>d.MoveNext() --- End of stack trace from previous location --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp2ConnectionAsync(HttpRequestMessage request) at System.Threading.Tasks.TaskCompletionSourceWithCancellation1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttp2ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at k8s.Kubernetes.ListNamespacedCustomObjectWithHttpMessagesAsync(String group, String version, String namespaceParameter, String plural, Nullable1 allowWatchBookmarks, String continueParameter, String fieldSelector, String labelSelector, Nullable1 limit, String resourceVersion, String resourceVersionMatch, Nullable1 timeoutSeconds, Nullable1 watch, Nullable1 pretty, Dictionary2 customHeaders, CancellationToken cancellationToken) at k8s.KubernetesExtensions.ListNamespacedCustomObjectAsync(IKubernetes operations, String group, String version, String namespaceParameter, String plural, Nullable1 allowWatchBookmarks, String continueParameter, String fieldSelector, String labelSelector, Nullable1 limit, String resourceVersion, String resourceVersionMatch, Nullable1 timeoutSeconds, Nullable1 watch, Nullable1 pretty, CancellationToken cancellationToken) at Orleans.Clustering.Kubernetes.KubeGatewayListProvider.GetGateways() at Orleans.Messaging.GatewayManager.RefreshSnapshotLiveGatewaysTimerCallback(Object context) in //src/Orleans.Core/Messaging/GatewayManager.cs:line 257`