Artifact Manager on S3 - issue with loading links to the S3 artifacts

davorceman commented 4 months ago

Jenkins and plugins versions report

Environment

```text Jenkins: 2.452.1 OS: Linux - 5.10.215-203.850.amzn2.aarch64 --- artifact-manager-s3:871.v72f7f642a_245 configuration-as-code:1810.v9b_c30a_249a_4c ... ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Jenkins: 2.452.1 OS: Linux - 5.10.215-203.850.amzn2.aarch64

Reproduction steps

Build Docker Image
```
FROM jenkins/jenkins:lts
```

ENV JENKINS_HOME=/data/jenkins ENV CASC_JENKINS_CONFIG=/data/jenkins/casc_configs/ ENV JAVA_OPTS="\ -Djenkins.model.Jenkins.buildsDir=/data/jenkins/builds/\${ITEM_FULL_NAME} \ -Djava.awt.headless=true \ -Djenkins.install.runSetupWizard=false \ -Dorg.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=true \ -Dlog4j2.formatMsgNoLookups=true \ -Dio.jenkins.plugins.artifact_manager_jclouds.s3.S3BlobStoreConfig.deleteArtifacts=true \ -Dio.jenkins.plugins.artifact_manager_jclouds.s3.S3BlobStoreConfig.deleteStashes=true"

USER root RUN mkdir -p /data/jenkins/jobs /data/jenkins/builds && \ chown -R jenkins:jenkins /data/jenkins USER jenkins

COPY plugins.txt /usr/share/jenkins/ref/plugins.txt RUN jenkins-plugin-cli --plugin-file /usr/share/jenkins/ref/plugins.txt

COPY casc_configs/ /data/jenkins/casc_configs/

EXPOSE 8080


JCasC:

aws: awsCredentials: region: "eu-west-2" s3: container: "my-s3-bucket" disableSessionToken: false useHttp: false usePathStyleUrl: false useTransferAcceleration: false unclassified: artifactManager: artifactManagerFactories:

jclouds: provider: "s3"


2. Run container in ECS Fargate with mounted jobs and builds directories from EFS. These jobs/builds are created with some another Jenkins instance, running in EC2, this will be migration to containerized Jenkins. ECS is behind ALB. ECS Task role has access to S3 bucket.
3. Login into Jenkins UI, open some job, "Last build Artifacts" url is missing. Open some build, artifacts are missing
4. Go to Mange -> AWS page, test connection to S3, result is 'success'. Just save page without any changes, and builds will start showing proper urls to Artifacts

Expected Results

To have artifacts links in jobs without manual saving Manage Jenkins -> AWS page

Actual Results

URLs to the Artifacts are missing

Anything else?

I have Jenkins instance which is running in EC2 for a long time. I want to start using Jenkins Docker together with CasC. ECS task has mounted builds and jobs directories from EFS, and those are properly loaded in my now docker jenkins, but Artifacts are missing. I can fix this just by save AWS config page in Jenkins, without any changes.

I have jenkins in non standard directory such as /data/jenkins, and also build directory path is not default

timja commented 4 months ago

Can you raise this with the artifact manager with S3 plugin please, its unlikely to be an issue with configuration as code

davorceman commented 4 months ago

@timja I think I just figured out

Seems that prefix parameter is required even value is empty string. I don't have prefix, all artifacts are in the root of the bucket.

I compared io.jenkins.plugins.artifact_manager_jcloud.s3.S3BlobstorageConfig.xml before and after save, and after saving few lines are added to that .xml

<prefix></prefix>
<customEndpoint></customEndpoint>
<customSigningRegion></customSigningRegion>

I did not set it because when I checked "View Configuration", prefix parameter was omitted. Also now I noticed that my new builds without prefix set created directory started with null. So instead having

s3://my-bucket-name/my-job-name/master/425/artifacts/....

this was created

s3://my-bucket-name/nullmy-job-name/master/425/artifacts/....

So this CasC solved the issue

aws:
  awsCredentials:
    region: "eu-west-2"
  s3:
    container: "my-s3-bucket"
    disableSessionToken: false
    useHttp: false
    usePathStyleUrl: false
    useTransferAcceleration: false
    prefix: ""

Even I have now set prefix to empty string, CasC plugin on "View Configuration" did not added it there.

Do you still think this should be raised with Artifact Manager on S3 plugn?

timja commented 4 months ago

yes it should handle that case

jenkinsci / configuration-as-code-plugin