Upgrading to v3.0.0: Scoped service of Container id providers got captured by singleton of container id holder - breaks at runtime when scope validation is on

[EPinci]

Hi, got a weird one in here. Upgrading from v2.0.6 to v3.0.0 seems to break tests in Linux only (?).

I have a solution with multiple projects and tests. After upgrading, al test run fine on my visualstudio on Windows. When I commit the code and my pipeline executes them in a Linux based containerized agent, all tests fail like this:

[xUnit.net 00:00:03.47]     BCReports.Test.Integrations.HealthzTest.GetHealthz [FAIL]
  Failed BCReports.Test.Integrations.HealthzTest.GetHealthz [1 ms]
  Error Message:
   System.AggregateException : Some services are not able to be constructed (Error while validating the service descriptor 'ServiceType: Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder Lifetime: Singleton ImplementationType: Microsoft.ApplicationInsights.Kubernetes.Containers.ContainerIdHolder': Cannot consume scoped service 'System.Collections.Generic.IEnumerable`1[Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerIdProvider]' from singleton 'Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder'.) (Error while validating the service descriptor 'ServiceType: Microsoft.ApplicationInsights.Kubernetes.Pods.IPodInfoManager Lifetime: Scoped ImplementationType: Microsoft.ApplicationInsights.Kubernetes.Pods.PodInfoManager': Cannot consume scoped service 'System.Collections.Generic.IEnumerable`1[Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerIdProvider]' from singleton 'Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder'.) (Error while validating the service descriptor 'ServiceType: Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerStatusManager Lifetime: Scoped ImplementationType: Microsoft.ApplicationInsights.Kubernetes.Containers.ContainerStatusManager': Cannot consume scoped service 'System.Collections.Generic.IEnumerable`1[Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerIdProvider]' from singleton 'Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder'.) (Error while validating the service descriptor 'ServiceType: Microsoft.ApplicationInsights.Kubernetes.IK8sEnvironmentFactory Lifetime: Scoped ImplementationType: Microsoft.ApplicationInsights.Kubernetes.K8sEnvironmentFactory': Cannot consume scoped service 'System.Collections.Generic.IEnumerable`1[Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerIdProvider]' from singleton 'Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder'.)
---- System.InvalidOperationException : Error while validating the service descriptor 'ServiceType: Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder Lifetime: Singleton ImplementationType: Microsoft.ApplicationInsights.Kubernetes.Containers.ContainerIdHolder': Cannot consume scoped service 'System.Collections.Generic.IEnumerable`1[Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerIdProvider]' from singleton 'Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder'.
-------- System.InvalidOperationException : Cannot consume scoped service 'System.Collections.Generic.IEnumerable`1[Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerIdProvider]' from singleton 'Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder'.
---- System.InvalidOperationException : Error while validating the service descriptor 'ServiceType: Microsoft.ApplicationInsights.Kubernetes.Pods.IPodInfoManager Lifetime: Scoped ImplementationType: Microsoft.ApplicationInsights.Kubernetes.Pods.PodInfoManager': Cannot consume scoped service 'System.Collections.Generic.IEnumerable`1[Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerIdProvider]' from singleton 'Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder'.
-------- System.InvalidOperationException : Cannot consume scoped service 'System.Collections.Generic.IEnumerable`1[Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerIdProvider]' from singleton 'Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder'.
---- System.InvalidOperationException : Error while validating the service descriptor 'ServiceType: Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerStatusManager Lifetime: Scoped ImplementationType: Microsoft.ApplicationInsights.Kubernetes.Containers.ContainerStatusManager': Cannot consume scoped service 'System.Collections.Generic.IEnumerable`1[Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerIdProvider]' from singleton 'Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder'.
-------- System.InvalidOperationException : Cannot consume scoped service 'System.Collections.Generic.IEnumerable`1[Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerIdProvider]' from singleton 'Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder'.
---- System.InvalidOperationException : Error while validating the service descriptor 'ServiceType: Microsoft.ApplicationInsights.Kubernetes.IK8sEnvironmentFactory Lifetime: Scoped ImplementationType: Microsoft.ApplicationInsights.Kubernetes.K8sEnvironmentFactory': Cannot consume scoped service 'System.Collections.Generic.IEnumerable`1[Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerIdProvider]' from singleton 'Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder'.
-------- System.InvalidOperationException : Cannot consume scoped service 'System.Collections.Generic.IEnumerable`1[Microsoft.ApplicationInsights.Kubernetes.Containers.IContainerIdProvider]' from singleton 'Microsoft.ApplicationInsights.Kubernetes.IContainerIdHolder'.

Any idea?

Thank you

[xiaomi7732]

@EPinci, thanks for the report. By a brief look, it feels like a bug. Let me dive deeper and see what might cause the issue. At the same time, is it possible to share one of the failed unit tests with us? That might help locate the issue.

Thank you again for the report.

[EPinci]

Hey, I'm running almost plain vanilla integration tests per https://learn.microsoft.com/en-us/aspnet/core/test/integration-tests?view=aspnetcore-6.0

namespace Testing.Fixtures
{
    public class CustomWebApplicationFactoryFixture<TStartup> : WebApplicationFactory<TStartup> where TStartup: class
    {
        protected override void ConfigureWebHost(IWebHostBuilder builder)
        {
            builder.ConfigureServices(services =>
            {
                var descriptor = services.SingleOrDefault(
                    d => d.ServiceType == typeof(DbContextOptions<MyContext>));
                services.Remove(descriptor);

                services.AddDbContext<MyContext>(options => {
                    options.UseInMemoryDatabase("IntegrationTestsDB");
                }, optionsLifetime: ServiceLifetime.Singleton);

                TestDataUtils.InitializeContext(services);
            });
        }
    }
}

namespace API.Test.Integrations
{
    [Trait("Category", "Integration")]
    [Collection("WebAppCollection")]
    public class HealthzTest
    {
        private readonly HttpClient _client;

        public HealthzTest(CustomWebApplicationFactoryFixture<Startup> factory)
        {
            _client = factory.CreateClient();
        }

        [Fact]
        public async Task GetHealthz()
        {
            // The endpoint or route of the controller action.
            var httpResponse = await _client.GetAsync("/healthz");

            // Must be successful.
            httpResponse.EnsureSuccessStatusCode();
        }
    }
}

[xiaomi7732]

Hi @EPinci, thanks for the info. I think I found the issue. Working on a fix. Will post the PR here when it is ready for review.

[xiaomi7732]

Released in 3.1.0-beta1: https://www.nuget.org/packages/Microsoft.ApplicationInsights.Kubernetes/3.1.0-beta1. @EPinci, do you mind giving it a try?

Thanks.

[EPinci]

@xiaomi7732 This build looks good to me. Thank you!

[xiaomi7732]

I am glad to hear that. Thank you for the quick turnaround.