Bucket-only artifacts

[am-sebastiankunze]

Hello.

I am configuring an artifact repository following the documentation here as follows:

apiVersion: v1
kind: ConfigMap
metadata:
  annotations:
    workflows.argoproj.io/default-artifact-repository: default-artifact-repository
  name: argo-workflows-artifacts-repository
data:
  default-artifact-repository: |
    s3:
      bucket: default
      keyFormat: common/{{ workflow.name }}
      endpoint: argo-workflows-artifacts.infra.svc.cluster.local:9000
      insecure: true
      accessKeySecret:
        key: accesskey
        name: argo-workflows-artifacts
      secretKeySecret:
        key: secretkey
        name: argo-workflows-artifacts

Whenever I overwrite the key bucket as described here, the artifacts are still uploaded in bucket default.

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: hello-world
spec:
  artifactRepositoryRef:
    configMap: argo-workflows-artifacts-repository
  entrypoint: main
  templates:
    - name: main
      container:
        image: docker/whalesay:latest
        command: [sh, -c]
        args: ["cowsay hello world | tee /tmp/hello_world.txt"]
        volumeMounts:
          - name: tmp
            mountPath: /tmp
      volumes:
        - name: tmp
          emptyDir: {}
      outputs:
        artifacts:
          - name: hello_world
            path: /tmp/hello_world.txt
            s3:
              bucket: dev

I order to further debug it, I removed the key s3.bucket from my configMap and set it through the WorkFlowTemplate exclusively. This, however, return me the following error:

Error (exit code 1): failed to put file: Bucket name cannot be empty

How can I configure the bucket per WorkFlow or WorkFlowTemplate?

[alexec]

Can I please ask:

• Can you try this with a workflow, i.e. side-step the WorkflowTemplate.
• Attach logs

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[am-sebastiankunze]

I submitted the following WorkFlow in order to side-step the WorkFlowTemplate:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  name: hello-world
spec:
  artifactRepositoryRef:
    configMap: argo-workflows-artifacts-repository
  entrypoint: main
  templates:
    - name: main
      container:
        image: docker/whalesay:latest
        command: [sh, -c]
        args: ["cowsay hello world | tee /tmp/hello_world.txt"]
        volumeMounts:
          - name: tmp
            mountPath: /tmp
      volumes:
        - name: tmp
          emptyDir: {}
      outputs:
        artifacts:
          - name: hello_world
            path: /tmp/hello_world.txt
            s3:
              bucket: dev

But I am running into the same behavior as well.

time="2022-04-19T09:39:44.844Z" level=info msg="Processing workflow" namespace=infra workflow=hello-world
time="2022-04-19T09:39:44.848Z" level=info msg="Get configmaps 200"
time="2022-04-19T09:39:44.848Z" level=info msg="resolved artifact repository" artifactRepositoryRef="infra/argo-workflows-artifacts-repository#"
time="2022-04-19T09:39:44.848Z" level=info msg="Updated phase  -> Running" namespace=infra workflow=hello-world
time="2022-04-19T09:39:44.848Z" level=info msg="Pod node hello-world initialized Pending" namespace=infra workflow=hello-world
time="2022-04-19T09:39:44.855Z" level=info msg="Create events 201"
time="2022-04-19T09:39:44.890Z" level=info msg="Create pods 201"
time="2022-04-19T09:39:44.891Z" level=info msg="Created pod: hello-world (hello-world)" namespace=infra workflow=hello-world
time="2022-04-19T09:39:44.891Z" level=info msg="TaskSet Reconciliation" namespace=infra workflow=hello-world
time="2022-04-19T09:39:44.891Z" level=info msg=reconcileAgentPod namespace=infra workflow=hello-world
time="2022-04-19T09:39:44.906Z" level=info msg="Update workflows 200"
time="2022-04-19T09:39:44.906Z" level=info msg="Workflow update successful" namespace=infra phase=Running resourceVersion=376061557 workflow=hello-world
...
time="2022-04-19T09:39:54.890Z" level=info msg="Processing workflow" namespace=infra workflow=hello-world
time="2022-04-19T09:39:54.891Z" level=info msg="Pod failed: Error (exit code 1): failed to put file: Bucket name cannot be empty" displayName=hello-world namespace=infra pod=hello-world templateName=main workflow=hello-world
time="2022-04-19T09:39:54.891Z" level=info msg="Updating node hello-world exit code 0" namespace=infra workflow=hello-world
time="2022-04-19T09:39:54.891Z" level=info msg="Setting node hello-world outputs: {\"artifacts\":[{\"name\":\"hello_world\",\"path\":\"/tmp/hello_world.txt\",\"s3\":{\"bucket\":\"dev\"}}]}" namespace=infra workflow=hello-world
time="2022-04-19T09:39:54.891Z" level=info msg="Updating node hello-world status Pending -> Error" namespace=infra workflow=hello-world
time="2022-04-19T09:39:54.891Z" level=info msg="Updating node hello-world message: Error (exit code 1): failed to put file: Bucket name cannot be empty" namespace=infra workflow=hello-world
time="2022-04-19T09:39:54.891Z" level=info msg="TaskSet Reconciliation" namespace=infra workflow=hello-world
time="2022-04-19T09:39:54.891Z" level=info msg=reconcileAgentPod namespace=infra workflow=hello-world
time="2022-04-19T09:39:54.891Z" level=info msg="Updated phase Running -> Error" namespace=infra workflow=hello-world
time="2022-04-19T09:39:54.891Z" level=info msg="Updated message  -> Error (exit code 1): failed to put file: Bucket name cannot be empty" namespace=infra workflow=hello-world
time="2022-04-19T09:39:54.891Z" level=info msg="Marking workflow completed" namespace=infra workflow=hello-world
time="2022-04-19T09:39:54.891Z" level=info msg="Checking daemoned children of " namespace=infra workflow=hello-world
time="2022-04-19T09:39:54.896Z" level=info msg="Create events 201"
time="2022-04-19T09:39:54.899Z" level=info msg="Update workflows 200"
time="2022-04-19T09:39:54.900Z" level=info msg="Workflow update successful" namespace=infra phase=Error resourceVersion=376062481 workflow=hello-world
time="2022-04-19T09:39:54.900Z" level=info msg="Queueing Error workflow infra/hello-world for delete in 24h0m0s"
time="2022-04-19T09:39:54.900Z" level=info msg="Queueing Error workflow infra/hello-world for delete in 24h0m0s"
...
time="2022-04-19T09:39:59.901Z" level=info msg="cleaning up pod" action=deletePod key=infra/hello-world/deletePod

Looks like the controller receives the parameter but fails to use it correctly.

Cheers, Sebastian

[alexec]

Ok. The issue here is that you MUST set the key. You cannot only set the bucket.

[tooptoop4]

dupe of https://github.com/argoproj/argo-workflows/issues/4349 ?