search

apache / camel-k

Apache Camel K is a lightweight integration platform, born on Kubernetes, with serverless superpowers
https://camel.apache.org/camel-k
Apache License 2.0
848 stars 345 forks source link

fatal error: concurrent map read and map write #5315

Closed tdiesler closed 2 months ago

tdiesler commented 3 months ago

What happened?

Multiple test may fail in the wake of such a concurrent map read/write

https://github.com/apache/camel-k/actions/runs/8521680570/job/23379943610?pr=5299

❌ TestSyntheticIntegrationFromDeployment (0s)
  Copy catalog camel-catalog-3.8.1 from namespace default
  Copy integration kit kit-co6gbjjd35ls73esvsp0 from namespace default
  Copy integration kit kit-co6gcrbd35ls73esvspg from namespace default
  Copy integration kit kit-co6gd03d35ls73esvsq0 from namespace default
  fatal error: concurrent map read and map write

  goroutine 627 [running]:
  k8s.io/apimachinery/pkg/runtime.(*Scheme).AddKnownTypeWithName(0xc0004df420, {{0x0, 0x0}, {0x271d9c7, 0x2}, {0x20d77d9, 0xb}}, {0x2d8c730?, 0xc000e180a0})
    /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.27.7/pkg/runtime/scheme.go:170 +0x1d7
  k8s.io/apimachinery/pkg/runtime.(*Scheme).AddKnownTypes(0xc0004df420, {{0x0?, 0x46f9d3?}, {0x271d9c7?, 0xc000775460?}}, {0xc0007755f0?, 0x5, 0xc00086e4b0?})
    /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.27.7/pkg/runtime/scheme.go:148 +0x176
  k8s.io/apimachinery/pkg/runtime.(*Scheme).AddUnversionedTypes(0xc0004df420, {{0x0?, 0x13?}, {0x271d9c7?, 0xa?}}, {0xc0007755f0?, 0x5, 0x5?})
    /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.27.7/pkg/runtime/scheme.go:124 +0xaf
  k8s.io/apimachinery/pkg/apis/meta/v1.AddToGroupVersion(0xc0004df420?, {{0x273b4c9?, 0x272323e?}, {0x271d9c7?, 0x27283c9?}})
    /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.27.7/pkg/apis/meta/v1/register.go:75 +0x335
  knative.dev/serving/pkg/apis/serving/v1.addKnownTypes(0x2db0600?)
    /home/runner/go/pkg/mod/knative.dev/serving@v0.39.3/pkg/apis/serving/v1/register.go:59 +0x20b
  k8s.io/apimachinery/pkg/runtime.(*SchemeBuilder).AddToScheme(...)
    /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.27.7/pkg/runtime/scheme_builder.go:29
  k8s.io/apimachinery/pkg/runtime.(*SchemeBuilder).AddToScheme(...)
    /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.27.7/pkg/runtime/scheme_builder.go:29
  github.com/apache/camel-k/v2/pkg/apis.AddToScheme(0xc000e955e0?)
    /home/runner/work/camel-k/camel-k/pkg/apis/apis.go:29 +0x5c
  github.com/apache/camel-k/v2/pkg/client.NewClientWithConfig(0x1, 0xc0008ac000)
    /home/runner/work/camel-k/camel-k/pkg/client/client.go:131 +0x3d
  github.com/apache/camel-k/v2/pkg/client.NewClient(0xb?)
    /home/runner/work/camel-k/camel-k/pkg/client/client.go:123 +0x45
  github.com/apache/camel-k/v2/pkg/client.NewOutOfClusterClient({0xc0007bac0b?, 0xc000775948?})
    /home/runner/work/camel-k/camel-k/pkg/client/client.go:114 +0x25
  github.com/apache/camel-k/v2/pkg/cmd.(*RootCmdOptions).NewCmdClient(...)
    /home/runner/work/camel-k/camel-k/pkg/cmd/root.go:248
  github.com/apache/camel-k/v2/pkg/cmd.(*RootCmdOptions).GetCmdClient(0xc00023a7e0)
    /home/runner/work/camel-k/camel-k/pkg/cmd/root.go:233 +0x2c
  github.com/apache/camel-k/v2/pkg/cmd.(*RootCmdOptions).preRun(0xc00023a7e0, 0xc0001cc300, {0x43bfdb?, 0xc0007f5700?, 0xc0001cc300?})
    /home/runner/work/camel-k/camel-k/pkg/cmd/root.go:179 +0x17b
  github.com/spf13/cobra.(*Command).execute(0xc0001cc300, {0xc000d87440, 0x3, 0x3})
    /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:954 +0x94b
  github.com/spf13/cobra.(*Command).ExecuteC(0xc000004600)
    /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115 +0x3ff
  github.com/spf13/cobra.(*Command).Execute(...)
    /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
  github.com/apache/camel-k/v2/e2e/advanced.TestRunIncrementalBuildWithDifferentBaseImages.func1({0x2da7e90?, 0x4384c60}, 0xc000901080, {0xc0008da7e0, 0x29})
    /home/runner/work/camel-k/camel-k/e2e/advanced/incremental_build_test.go:228 +0xf8d
  github.com/apache/camel-k/v2/e2e/support.invokeUserTestCode(0x539d8c?, {0x2da7e90?, 0x4384c60}, {0xc0008da7e0, 0x29}, 0xc000631f50)
    /home/runner/work/camel-k/camel-k/e2e/support/test_support.go:2881 +0xd9
  github.com/apache/camel-k/v2/e2e/support.WithNewTestNamespace(0xc000a5d380, 0x0?)
    /home/runner/work/camel-k/camel-k/e2e/support/test_support.go:2822 +0x107
  github.com/apache/camel-k/v2/e2e/advanced.TestRunIncrementalBuildWithDifferentBaseImages(0xc000a5d380)
    /home/runner/work/camel-k/camel-k/e2e/advanced/incremental_build_test.go:185 +0x3e
tdiesler commented 2 months ago

Done

tdiesler commented 2 months ago

Reopen, because of ...

  fatal error: concurrent map read and map write

  goroutine 15404 [running]:
  k8s.io/apimachinery/pkg/runtime.(*Scheme).ObjectKinds(0xc0002ae540, {0x2e5ddb0?, 0xc0004a2930})
    /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.28.8/pkg/runtime/scheme.go:267 +0x1ad
  k8s.io/apimachinery/pkg/runtime/serializer/protobuf.(*Serializer).Decode(0xc0008d5140, {0xc00140a000?, 0x1010f?, 0x12000?}, 0x0, {0x2e5ddb0?, 0xc0004a2930?})
    /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.28.8/pkg/runtime/serializer/protobuf/protobuf.go:132 +0x458
  k8s.io/apimachinery/pkg/runtime.WithoutVersionDecoder.Decode({{0x7f39dc397fd8?, 0xc0008d5140?}}, {0xc00140a000?, 0x199?, 0xc0004a2930?}, 0x259db80?, {0x2e5ddb0?, 0xc0004a2930?})
    /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.28.8/pkg/runtime/helper.go:252 +0x4f
  sigs.k8s.io/controller-runtime/pkg/client/apiutil.targetZeroingDecoder.Decode({{0x2e54ea0?, 0xc00107c050?}}, {0xc00140a000, 0x1010f, 0x12000}, 0x0?, {0x2e5ddb0?, 0xc0004a2930?})
    /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.5/pkg/client/apiutil/apimachinery.go:236 +0xb0
  k8s.io/client-go/rest.Result.Into({{0xc00140a000, 0x1010f, 0x12000}, {0x0, 0x0, 0x0}, {0xc000144060, 0x23}, {0x0, 0x0}, ...}, ...)
    /home/runner/go/pkg/mod/k8s.io/client-go@v0.28.8/rest/request.go:1373 +0x97
  sigs.k8s.io/controller-runtime/pkg/client.(*typedClient).List(0xc0008e8900, {0x2e7a818, 0x44b0940}, {0x2e87cc0?, 0xc0004a2930}, {0xc0009ba9c0, 0x2, 0x3655?})
    /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.5/pkg/client/typed_client.go:165 +0x2a5
  sigs.k8s.io/controller-runtime/pkg/client.(*client).List(0xc0008e8900, {0x2e7a818, 0x44b0940}, {0x2e87cc0?, 0xc0004a2930?}, {0xc0009ba9c0, 0x2, 0x2})
    /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.5/pkg/client/client.go:402 +0x3e5
  github.com/apache/camel-k/v2/e2e/support.IntegrationPod.func1.IntegrationPods.func1()
    /home/runner/work/camel-k/camel-k/e2e/support/test_support.go:649 +0x271
  github.com/apache/camel-k/v2/e2e/support.IntegrationPod.func1()
    /home/runner/work/camel-k/camel-k/e2e/support/test_support.go:623 +0x7d
  github.com/apache/camel-k/v2/e2e/advanced.TestKamelCLIPromote.func1.4.8.IntegrationLogs.func6()

This code cannot be called concurrently 

err := TestClient(t).List(ctx, &lst,
    ctrl.InNamespace(ns),
    ctrl.MatchingLabels{
        v1.IntegrationLabel: name,
    })
if err != nil {
    failTest(t, err)
}
tdiesler commented 2 months ago

Reported here: https://github.com/kubernetes-sigs/controller-runtime/issues/2782

tdiesler commented 2 months ago

Done