search

crossplane-contrib / function-kcl

Crossplane Composition Functions using KCL Programming Language
Apache License 2.0
36 stars 16 forks source link

Infinite /tmp/sandbox folders #163

Open jgascon-nx opened 5 hours ago

jgascon-nx commented 5 hours ago

What happened?

When running function-kcl v0.9.4 and using it in a composition as inline code, there comes a time where you can end up without space left on device. This happens because the function-kcl reaches this point in the code( which uses the krm-kcl lib) and this one eventually reaches to the line that creates the folder.

The issue is that these folders are not cleaned up ever, so they remain there forever til the pod eventually gets blocked repoducing issues like the following one in a CompositeResource:

Events:
  Type     Reason            Age                        From                                                             Message
  ----     ------            ----                       ----                                                             -------
  Warning  ComposeResources  72s (x1220837 over 2d22h)  defined/compositeresourcedefinition.apiextensions.crossplane.io  (combined from similar events): cannot compose resources: pipeline step "sqs-queue" returned a fatal result: failed to run kcl function pipelines: error creating temp directory: mkdir /tmp/sandbox2528510530: no space left on device

Is there a way to clean this up from within the function-kcl or this requires to adapt krm-kcl? Could it be that I'm doing something wrong? It seems the Function pod ends up being highly unstable and bringing issues in the mid-long term.

How can we reproduce it?

kubectl exec -it function-kcl-0bd316c25ddd-6f4666f578-w4cwr -- ls -lah /tmp | tail
# ...
# drwx------ 2 2000 2000  20 Sep 30 14:52 sandbox999888316
# drwx------ 2 2000 2000  20 Sep 30 13:46 sandbox999908985
# drwx------ 2 2000 2000  20 Sep 30 14:58 sandbox99991071
# drwx------ 2 2000 2000  20 Sep 30 12:51 sandbox999919502
# drwx------ 2 2000 2000  20 Sep 30 14:50 sandbox999921334
# drwx------ 2 2000 2000  20 Sep 30 13:08 sandbox999932367
# drwx------ 2 2000 2000  20 Sep 30 15:45 sandbox99993400
# drwx------ 2 2000 2000  20 Sep 30 15:08 sandbox999958027
# drwx------ 2 2000 2000  20 Sep 30 14:19 sandbox999992460
# drwx------ 2 2000 2000  20 Sep 30 13:44 sandbox999994535

kubectl exec -it function-kcl-0bd316c25ddd-6f4666f578-w4cwr -- ls -lah /tmp | wc -l
# 353577
# ^ Increasing forever

kubectl exec -it function-kcl-0bd316c25ddd-6f4666f578-w4cwr -- du -sh /tmp/
# 2.8G    /tmp/
# ^ Increasing forever

What environment did it happen in?

Function version: v0.9.4 Function image: kcl-arm64

Peefy commented 3 hours ago

Sorry, this is indeed a bug in KRM KCL, the temp folder can be removed after the function is executed.