search

crossplane-contrib / provider-upjet-aws

Official AWS Provider for Crossplane by Upbound.
https://marketplace.upbound.io/providers/upbound/provider-aws
Apache License 2.0
142 stars 121 forks source link

[Bug]: fatal error: concurrent map writes in ECS Task Definition #1426

Open daniel-maganto opened 1 month ago

daniel-maganto commented 1 month ago

Is there an existing issue for this?

Affected Resource(s)

taskdefinition.ecs.aws.upbound.io/v1beta1

Resource MRs required to reproduce the bug

apiVersion: ecs.aws.upbound.io/v1beta1
kind: TaskDefinition
metadata:
  annotations:
    crossplane.io/composition-resource-name: taskdefinition
    crossplane.io/external-create-pending: "2024-04-23T07:12:22Z"
    crossplane.io/external-create-succeeded: "2024-04-23T07:12:22Z"
    crossplane.io/external-name: my-service
  creationTimestamp: "2024-04-23T07:12:08Z"
  finalizers:
  - finalizer.managedresource.crossplane.io
  generateName: my-service-dddddddddd-nj6dk-
  generation: 4
  labels:
    crossplane.io/claim-name: my-service-dddddddddd
    crossplane.io/claim-namespace: service-crossplane-test
    crossplane.io/composite: my-service-dddddddddd-nj6dk
    taskdefinition: my-service-my-service-dddddddddd
    tenant: my-service
  name: my-service-test-dddddddddd
  ownerReferences:
spec:
  deletionPolicy: Delete
  forProvider:
    cpu: "256"
    executionRoleArn: arn:aws:iam::1234567123456:role/my-service-test-executionrole
    executionRoleArnRef:
      name: my-service-test-executionrole
    executionRoleArnSelector:
      matchLabels:
        role: my-service-test-executionrole
    family: my-service
    memory: "512"
    networkMode: awsvpc
    region: eu-central-1
    requiresCompatibilities:
    - FARGATE
    tags:
      crossplane-kind: taskdefinition.ecs.aws.upbound.io
      crossplane-name: my-service-test-dddddddddd
      crossplane-providerconfig: aws-test
    taskRoleArn: arn:aws:iam::1234567123456:role/my-service-test-taskrole
  initProvider:
    containerDefinitions: [  \n  {\n    \"command\": [],\n    \"cpu\": 0,\n    \"environment\":
      [\n      {\n        \"name\": \"STAGE\",\n        \"value\": \"test\"\n      },\n
      \     {\n        \"name\": \"TENANT\",\n        \"value\": \"my-service\"\n
      \     },\n      {\n        \"name\": \"OTLP_ENABLED\",\n        \"value\": \"true\"\n
      \     },\n      {\n        \"name\": \"OTLP_ENDPOINT\",\n        \"value\":
      \"http://my-collector\"\n
      \     }\n    ],\n    \"essential\": true,\n    \"image\": \"1234567123456.dkr.ecr.eu-central-1.amazonaws.com/test/my-service:v0.0.2-cputest\",\n
      \   \"logConfiguration\": {\n      \"logDriver\": \"awslogs\",\n      \"options\":
      {\n        \"awslogs-group\": \"my-service-test\",\n        \"awslogs-region\":
      \"eu-central-1\",\n        \"awslogs-stream-prefix\": \"my-service\"\n      }\n
      \   },\n    \"name\": \"my-service\",\n    \"portMappings\": [\n      {\n        \"containerPort\":
      8080,\n        \"hostPort\": 8080,\n        \"protocol\": \"tcp\"\n      }\n
      \   ],\n    \"secrets\": []\n  }\n]
  managementPolicies:
  - Observe
  - Create
  - Update
  - Delete
  providerConfigRef:
    name: aws-test

Steps to Reproduce

When you create the resource, if the ECS provider pod restarts due the error, it creates automatically another iteration of the task definition in an uncontrolled behaviour

What happened?

it creates automatically another iteration of the task definition in an uncontrolled behaviour

Relevant Error Output Snippet

2024-07-29T09:51:32Z    DEBUG   provider-aws    Async update starting...    {"trackerUID": "b75dcb54-5860-4196-9d6d-3091994b631e", "resourceName": "myservice-policy-preprod", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service", "tfID": "arn:aws:ecs:eu-central-1:695964746098:service/myservice-preprod/myservice-policy-preprod"}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Updating the external resource  {"uid": "b75dcb54-5860-4196-9d6d-3091994b631e", "name": "myservice-policy-preprod", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service"}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Reconcile request has been requeued.    {"gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service", "name": "myservice-policy-test", "rateLimiterName": "", "when": "0s"}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Reconciling {"controller": "managed/ecs.aws.upbound.io/v1beta1, kind=service", "request": {"name":"myservice-policy-test"}}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Connecting to the service provider  {"uid": "cfb18461-a50a-4822-b2eb-4597836fe5db", "name": "myservice-policy-prod", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service"}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Reconciling {"controller": "managed/ecs.aws.upbound.io/v1beta1, kind=service", "request": {"name":"devops-service-dev"}}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Observing the external resource {"uid": "cfb18461-a50a-4822-b2eb-4597836fe5db", "name": "myservice-policy-prod", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service"}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Connecting to the service provider  {"uid": "ca0e78a5-36f2-4a3f-bcc7-d40fb2099879", "name": "myservice-policy-test", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service"}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Async update ended. {"trackerUID": "b75dcb54-5860-4196-9d6d-3091994b631e", "resourceName": "myservice-policy-preprod", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service", "error": null, "tfID": "arn:aws:ecs:eu-central-1:695964746098:service/myservice-preprod/myservice-policy-preprod"}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Diff detected   {"uid": "cfb18461-a50a-4822-b2eb-4597836fe5db", "name": "myservice-policy-prod", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service", "instanceDiff": "*terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{\"task_definition\":*terraform.ResourceAttrDiff{Old:\"myservice-policy:87\", New:\"myservice-policy\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, RawConfig:cty.NilVal, RawState:cty.NilVal, RawPlan:cty.NilVal, Meta:map[string]interface {}(nil)}"}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Successfully requested update of external resource  {"controller": "managed/ecs.aws.upbound.io/v1beta1, kind=service", "request": {"name":"myservice-policy-prod"}, "uid": "cfb18461-a50a-4822-b2eb-4597836fe5db", "version": "537236095", "external-name": "myservice-policy-prod", "requeue-after": "2024-07-29T10:01:31Z"}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Async update starting...    {"trackerUID": "cfb18461-a50a-4822-b2eb-4597836fe5db", "resourceName": "myservice-policy-prod", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service", "tfID": "arn:aws:ecs:eu-central-1:459316410691:service/myservice-prod/myservice-policy-prod"}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Updating the external resource  {"uid": "cfb18461-a50a-4822-b2eb-4597836fe5db", "name": "myservice-policy-prod", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service"}
2024-07-29T09:51:32Z    DEBUG   provider-aws    Observing the external resource {"uid": "ca0e78a5-36f2-4a3f-bcc7-d40fb2099879", "name": "myservice-policy-test", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service"}
2024-07-29T09:51:33Z    DEBUG   provider-aws    Diff detected   {"uid": "ca0e78a5-36f2-4a3f-bcc7-d40fb2099879", "name": "myservice-policy-test", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service", "instanceDiff": "*terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{\"task_definition\":*terraform.ResourceAttrDiff{Old:\"myservice-policy:807\", New:\"myservice-policy\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, RawConfig:cty.NilVal, RawState:cty.NilVal, RawPlan:cty.NilVal, Meta:map[string]interface {}(nil)}"}
2024-07-29T09:51:33Z    DEBUG   provider-aws    Successfully requested update of external resource  {"controller": "managed/ecs.aws.upbound.io/v1beta1, kind=service", "request": {"name":"myservice-policy-test"}, "uid": "ca0e78a5-36f2-4a3f-bcc7-d40fb2099879", "version": "535933590", "external-name": "myservice-policy-test", "requeue-after": "2024-07-29T10:01:54Z"}
2024-07-29T09:51:33Z    DEBUG   provider-aws    Async update starting...    {"trackerUID": "ca0e78a5-36f2-4a3f-bcc7-d40fb2099879", "resourceName": "myservice-policy-test", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service", "tfID": "arn:aws:ecs:eu-central-1:784471420242:service/myservice-test/myservice-policy-test"}
2024-07-29T09:51:33Z    DEBUG   provider-aws    Updating the external resource  {"uid": "ca0e78a5-36f2-4a3f-bcc7-d40fb2099879", "name": "myservice-policy-test", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service"}
2024-07-29T09:51:33Z    DEBUG   provider-aws    Async update ended. {"trackerUID": "cfb18461-a50a-4822-b2eb-4597836fe5db", "resourceName": "myservice-policy-prod", "gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service", "error": null, "tfID": "arn:aws:ecs:eu-central-1:459316410691:service/myservice-prod/myservice-policy-prod"}
2024-07-29T09:51:33Z    DEBUG   provider-aws    Reconcile request has been requeued.    {"gvk": "ecs.aws.upbound.io/v1beta1, Kind=Service", "name": "myservice-policy-preprod", "rateLimiterName": "", "when": "0s"}
2024-07-29T09:51:33Z    DEBUG   provider-aws    Reconciling {"controller": "managed/ecs.aws.upbound.io/v1beta1, kind=service", "request": {"name":"myservice-policy-preprod"}}
fatal error: concurrent map writes

goroutine 25476453 [running]:
reflect.mapassign0(0xffa3d80, 0xc002c26ca0?, 0xc00db71f48?, 0xc00c190a01?)
    runtime/map.go:1370 +0x1d
reflect.mapassign(0xc00c190ab0?, 0xc00235fb80?, 0xc00235fbd0?, 0x4101c9?)
    reflect/value.go:3828 +0x8d
github.com/modern-go/reflect2.(*UnsafeMapType).UnsafeSetIndex(...)
    github.com/modern-go/reflect2@v1.0.2/unsafe_map.go:76
github.com/json-iterator/go.(*mapDecoder).Decode(0xc0010ee6e0, 0xc00b0c9720, 0xc00c190ab0)
    github.com/json-iterator/go@v1.1.12/reflect_map.go:191 +0x325
github.com/json-iterator/go.(*placeholderDecoder).Decode(0xc002c28c00?, 0x415c0b?, 0xc00235fc58?)
    github.com/json-iterator/go@v1.1.12/reflect.go:324 +0x1b
github.com/json-iterator/go.(*structFieldDecoder).Decode(0xc002c4dd20, 0xc0036bcf30?, 0xc00c190ab0)
    github.com/json-iterator/go@v1.1.12/reflect_struct_decoder.go:1054 +0x50
github.com/json-iterator/go.(*generalStructDecoder).decodeOneField(0xc002c4de00, 0xc001e92a90?, 0xc00c190ab0)
    github.com/json-iterator/go@v1.1.12/reflect_struct_decoder.go:552 +0x29c
github.com/json-iterator/go.(*generalStructDecoder).Decode(0xc002c4de00, 0xf222700?, 0xc00c190ab0)
    github.com/json-iterator/go@v1.1.12/reflect_struct_decoder.go:508 +0x8b
github.com/json-iterator/go.(*Iterator).ReadVal(0xc00c190ab0, {0x1043c740, 0xc00b0c95b8})
    github.com/json-iterator/go@v1.1.12/reflect.go:79 +0x123
github.com/json-iterator/go.(*frozenConfig).Unmarshal(0xc0005c46e0, {0xc0021d8000?, 0xc004332ae0?, 0x11efa4a0?}, {0x1043c740, 0xc00b0c95b8})
    github.com/json-iterator/go@v1.1.12/config.go:348 +0x99
github.com/upbound/provider-aws/apis/ecs/v1beta1.(*Service).SetObservation(0xc00b0c9100, 0xc00235fef8?)
    github.com/upbound/provider-aws/apis/ecs/v1beta1/zz_service_terraformed.go:43 +0x7e
github.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKExternal).Update(0xc00a9902a0, {0x14750220, 0xc007eccc40}, {0x147c8f08?, 0xc00b0c9100})
    github.com/crossplane/upjet@v1.3.0-rc.0.0.20240314111422-84abd051e678/pkg/controller/external_tfpluginsdk.go:678 +0x27b
github.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKAsyncExternal).Update.func1()
    github.com/crossplane/upjet@v1.3.0-rc.0.0.20240314111422-84abd051e678/pkg/controller/external_async_tfpluginsdk.go:170 +0x145
created by github.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKAsyncExternal).Update in goroutine 4050
    github.com/crossplane/upjet@v1.3.0-rc.0.0.20240314111422-84abd051e678/pkg/controller/external_async_tfpluginsdk.go:166 +0x168

goroutine 1 [select, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).Start(0xc000d03d40, {0x147501b0, 0xc009e594f0})
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/internal.go:444 +0x987
main.main()
    github.com/upbound/provider-aws/cmd/provider/ecs/zz_main.go:205 +0x34cc

goroutine 2448 [chan receive, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/manager/signals.SetupSignalHandler.func1()
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/signals/signal.go:38 +0x27
created by sigs.k8s.io/controller-runtime/pkg/manager/signals.SetupSignalHandler in goroutine 1
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/signals/signal.go:37 +0xc5

goroutine 25434898 [IO wait]:
internal/poll.runtime_pollWait(0x7fc3a13133d0, 0x72)
    runtime/netpoll.go:343 +0x85
internal/poll.(*pollDesc).wait(0xc00c766100?, 0xc005938000?, 0x0)
    internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
    internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc00c766100, {0xc005938000, 0x1800, 0x1800})
    internal/poll/fd_unix.go:164 +0x27a
net.(*netFD).Read(0xc00c766100, {0xc005938000?, 0xc005938005?, 0xae?})
    net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc000181990, {0xc005938000?, 0x12c?, 0xc00bc30538?})
    net/net.go:179 +0x45
crypto/tls.(*atLeastReader).Read(0xc003be8870, {0xc005938000?, 0xc003be8870?, 0x0?})
    crypto/tls/conn.go:805 +0x3b
bytes.(*Buffer).ReadFrom(0xc00bc30628, {0x145e9900, 0xc003be8870})
    bytes/buffer.go:211 +0x98
crypto/tls.(*Conn).readFromUntil(0xc00bc30380, {0x145ea4e0?, 0xc000181990}, 0x1800?)
    crypto/tls/conn.go:827 +0xde
crypto/tls.(*Conn).readRecordOrCCS(0xc00bc30380, 0x0)
    crypto/tls/conn.go:625 +0x250
crypto/tls.(*Conn).readRecord(...)
    crypto/tls/conn.go:587
crypto/tls.(*Conn).Read(0xc00bc30380, {0xc0034d1000, 0x1000, 0x22?})
    crypto/tls/conn.go:1369 +0x158
net/http.(*persistConn).Read(0xc005afec60, {0xc0034d1000?, 0xc00a0e33e0?, 0xc004799d38?})
    net/http/transport.go:1954 +0x4a
bufio.(*Reader).fill(0xc00b6266c0)
    bufio/bufio.go:113 +0x103
bufio.(*Reader).Peek(0xc00b6266c0, 0x1)
    bufio/bufio.go:151 +0x53
net/http.(*persistConn).readLoop(0xc005afec60)
    net/http/transport.go:2118 +0x1b9
created by net/http.(*Transport).dialConn in goroutine 25392746
    net/http/transport.go:1776 +0x169f

goroutine 999 [chan receive, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start(0xc009e6dc20, {0x147501b0, 0xc000f58d20})
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:239 +0x17b
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1(0xc00a572980)
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:223 +0xc8
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile in goroutine 46
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:207 +0x19d

goroutine 3569 [syscall, 345 minutes]:
os/signal.signal_recv()
    runtime/sigqueue.go:152 +0x29
os/signal.loop()
    os/signal/signal_unix.go:23 +0x13
created by os/signal.Notify.func1.1 in goroutine 1
    os/signal/signal.go:151 +0x1f

goroutine 3585 [chan receive, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile(0xc00024f440)
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:183 +0x45
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).Start.func1 in goroutine 1
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:136 +0xc8

goroutine 3586 [IO wait]:
internal/poll.runtime_pollWait(0x7fc3a156bfc8, 0x72)
    runtime/netpoll.go:343 +0x85
internal/poll.(*pollDesc).wait(0xc00a636000?, 0x4?, 0x0)
    internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
    internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc00a636000)
    internal/poll/fd_unix.go:611 +0x2ac
net.(*netFD).accept(0xc00a636000)
    net/fd_unix.go:172 +0x29
net.(*TCPListener).accept(0xc00a60a060)
    net/tcpsock_posix.go:152 +0x1e
net.(*TCPListener).Accept(0xc00a60a060)
    net/tcpsock.go:315 +0x30
net/http.(*Server).Serve(0xc00055ef00, {0x14741608, 0xc00a60a060})
    net/http/server.go:3056 +0x364
sigs.k8s.io/controller-runtime/pkg/metrics/server.(*defaultServer).Start(0xc00096f040, {0x147501b0?, 0xc000f58c30})
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/metrics/server/server.go:242 +0x848
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1(0xc00a60a020)
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:223 +0xc8
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile in goroutine 3585
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:207 +0x19d

goroutine 40 [chan receive, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile(0xc00024f4d0)
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:183 +0x45
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).Start.func1 in goroutine 1
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:136 +0xc8

goroutine 41 [chan receive, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile(0xc00024f560)
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:183 +0x45
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).Start.func1 in goroutine 1
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:136 +0xc8

goroutine 42 [chan receive, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/cache/internal.(*Informers).Start(0xc000b16780, {0x147501b0, 0xc000f58cd0})
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/cache/internal/informers.go:211 +0x3d
sigs.k8s.io/controller-runtime/pkg/cluster.(*cluster).Start(0xc003fc6178?, {0x147501b0?, 0xc000f58cd0?})
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/cluster/internal.go:104 +0x76
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1(0xc00a60a000)
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:223 +0xc8
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile in goroutine 41
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:207 +0x19d

goroutine 3570 [chan receive, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start(0xc009e6d9a0, {0x147501b0, 0xc000f58d20})
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:239 +0x17b
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1(0xc00a5723a0)
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:223 +0xc8
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile in goroutine 46
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:207 +0x19d

goroutine 44 [chan receive, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile(0xc00024f680)
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:183 +0x45
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).Start.func1 in goroutine 1
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:136 +0xc8

goroutine 46 [chan receive, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile(0xc00024f5f0)
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:183 +0x45
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).Start.func1 in goroutine 45
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:136 +0xc8

goroutine 47 [chan receive, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start(0xc009e6d860, {0x147501b0, 0xc000f58d20})
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:239 +0x17b
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1(0xc00a5720a0)
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:223 +0xc8
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile in goroutine 46
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:207 +0x19d

goroutine 58 [select]:
k8s.io/client-go/util/workqueue.(*delayingType).waitingLoop(0xc000f70ae0)
    k8s.io/client-go@v0.29.2/util/workqueue/delaying_queue.go:276 +0x305
created by k8s.io/client-go/util/workqueue.newDelayingQueue in goroutine 3570
    k8s.io/client-go@v0.29.2/util/workqueue/delaying_queue.go:113 +0x21f

goroutine 3571 [chan receive, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start(0xc009e6dae0, {0x147501b0, 0xc000f58d20})
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:239 +0x17b
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1(0xc00a5726a0)
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:223 +0xc8
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile in goroutine 46
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:207 +0x19d

goroutine 3589 [chan receive, 345 minutes]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start(0xc009e6dea0, {0x147501b0, 0xc000f58d20})
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:239 +0x17b
sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1(0xc00a572f60)
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:223 +0xc8
created by sigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile in goroutine 46
    sigs.k8s.io/controller-runtime@v0.17.2/pkg/manager/runnable_group.go:207 +0x19d

Crossplane Version

1.15.1

Provider Version

1.4.0

Kubernetes Version

v1.29.4-eks-036c24b

Kubernetes Distribution

EKS

Additional Info

No response

daniel-maganto commented 1 month ago

If it helps, checking the logs and behavior it seems that services always try to sync between the taskDefinition and can't.

I also updated to the latest version of the providers but got the same behavior.

apiVersion: ecs.aws.upbound.io/v1beta2
kind: Service
metadata:
[...]
spec:
  forProvider:
    taskDefinition: my-service-test
    taskDefinitionRef:
      name: my-service-test-dddddddddd
status:
  atProvider:
[...]
    taskDefinition: my-service-test:30
[...]