jenkins-infra / repository-permissions-updater

Artifactory permissions synchronization tool and data set
79 stars 1.05k forks source link

Hosting request for CloudAEye Plugin #4131

Closed bhavyalatha26 closed 2 weeks ago

bhavyalatha26 commented 1 month ago

Repository URL

https://github.com/CloudAEye/cloudaeye-plugin

New Repository Name

cloudaeye-plugin

Description

The CloudAEye Plugin is designed to enhance CI/CD pipelines by seamlessly exporting job details and build logs from Jenkins to the CloudAEye platform. CloudAEye analyzes test failure logs to identify root causes and suggests actionable code fixes, helping developers resolve issues more efficiently. By providing real-time insights into build failures, the plugin enables faster troubleshooting and minimizes downtime in software delivery pipelines.

Key Features:

GitHub users to have commit permission

@bhavyalatha26 @nazrul-cloudaeye @PiyushKumar11

Jenkins project users to have release permission

cloudaeye

Issue tracker

GitHub issues

jenkins-cert-app commented 1 month ago

Security audit, information and commands

The security team is auditing all the hosting requests, to ensure a better security by default.

This message informs you that a Jenkins Security Scan was triggered on your repository. It takes ~10 minutes to complete.

Commands The bot will parse all comments, and it will check if any line start with a command. Security team only:
  • /audit-ok => the audit is complete, the hosting can continue :tada:.
  • /audit-skip => the audit is not necessary, the hosting can continue :tada:.
  • /audit-findings => the audit reveals some issues that require corrections :pencil2:.
Anyone:
  • /request-security-scan => the findings from the Jenkins Security Scan were corrected, this command will re-scan your repository :mag:.
  • /audit-review => the findings from the audit were corrected, this command will ping the security team to review the findings :eyes:. It's only applicable when the previous audit required changes.
Only one command can be requested per comment.

(automatically generated message, version: 1.29.20)

github-actions[bot] commented 1 month ago

Hello from your friendly Jenkins Hosting Checker

It appears you have some issues with your hosting request. Please see the list below and correct all issues marked Required. Your hosting request will not be approved until these issues are corrected. Issues marked with Warning or Info are just recommendations and will not stall the hosting process.

You can re-trigger a check by editing your hosting request or by commenting /hosting re-check

jenkins-cert-app commented 1 month ago

The Jenkins Security Scan discovered 5 finding(s) :mag:.

For every identified issue, please do one of the following:

After addressing the findings through one of the above methods:


Stapler: Missing permission check

You can find detailed information about this finding here.

GlobalKeyConfiguration.java#91 ``` Potential missing permission check in GlobalKeyConfiguration#doTestConnection ```

Jenkins: Plaintext password storage

You can find detailed information about this finding here.

CloudAEyeNotifications.java#36 ``` Field should be reviewed whether it stores a password and is serialized to disk: token ```
CloudAEyeNotifications.java#34 ``` Field should be reviewed whether it stores a password and is serialized to disk: tenantKey ```
GlobalKeyConfiguration.java#35 ``` Field should be reviewed whether it stores a password and is serialized to disk: token ```
GlobalKeyConfiguration.java#34 ``` Field should be reviewed whether it stores a password and is serialized to disk: tenantKey ```
bhavyalatha26 commented 1 month ago

/hosting re-check

github-actions[bot] commented 1 month ago

Hello from your friendly Jenkins Hosting Checker

It appears you have some issues with your hosting request. Please see the list below and correct all issues marked Required. Your hosting request will not be approved until these issues are corrected. Issues marked with Warning or Info are just recommendations and will not stall the hosting process.

You can re-trigger a check by editing your hosting request or by commenting /hosting re-check

bhavyalatha26 commented 1 month ago

/request-security-scan

jenkins-cert-app commented 1 month ago

The Jenkins Security Scan did not find anything dangerous with your plugin, congratulations! :tada:


:bulb: The Security team recommends that you are setting up the scan in your repository by following our guide.

bhavyalatha26 commented 1 month ago

/hosting re-check

github-actions[bot] commented 1 month ago

Hello from your friendly Jenkins Hosting Checker

It looks like you have everything in order for your hosting request. A member of the Jenkins hosting team will check over things that I am not able to check(code review, README content, etc) and process the request as quickly as possible. Thank you for your patience.

Hosting team members can host this request with /hosting host

mawinter69 commented 1 month ago
mawinter69 commented 1 month ago
mawinter69 commented 1 month ago

Please provide a meaningful description here: https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/resources/index.jelly#L3 This is what is shown to users in the Jenkins UI when they look at the pluginmanager.

bhavyalatha26 commented 1 month ago

No our plugin for both free-style and pipeline, you can find it in implementation ( https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L198) and also in README instructions on how to use it pipeline

bhavyalatha26 commented 1 month ago

Please provide a meaningful description here: https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/resources/index.jelly#L3 This is what is shown to users in the Jenkins UI when they look at the pluginmanager.

Sure will update it.

bhavyalatha26 commented 1 month ago

@mawinter69 are there any other suggestions that we need to improve on so that we can finish all of them together at once and resubmit for review ?

mawinter69 commented 1 month ago

No our plugin for both free-style and pipeline, you can find it in implementation ( https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L198) and also in README instructions on how to use it pipeline

If you support pipeline you should annotate the descriptor with @Symbol('cloudAEyeNotification') (singular as you send only one request) at https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L321 This allows you to write in a pipeline cloudAEyeNotification() instead of step([$class: 'CloudAEyeNotifications']) Also change line 327 so that it works with Job instead of AbstractProject. Otherwise I think the step will not appear in the Snippet Generator.

The plugin seems to be focused on git especially github. There are many other SCMs around like gerrit, gitlab, bitbucket, perforce and many more

bhavyalatha26 commented 1 month ago

Please provide a meaningful description here: https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/resources/index.jelly#L3 This is what is shown to users in the Jenkins UI when they look at the pluginmanager.

Sure will update it.

Done

bhavyalatha26 commented 1 month ago

Committed most of the suggested changes. Here are few that we did not / failed to:

  1. [In Progress] Migrating to secrets for token

  2. Remove redundant check box: We want to keep this because in upcoming updates, we need this feature to help user select some preferences based on this

  3. Also change line 327 so that it works with Job instead of AbstractProject. Otherwise I think the step will not appear in the Snippet Generator.

    Changing it to Job was throwing error:

    Class 'DescriptorImpl' must either be declared abstract or implement abstract method 'isApplicable(Class<? extends AbstractProject>)' in 'BuildStepDescriptor'
  4. https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L99 is there a missing return statement? https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L132 same here missing return?

    Since it is a void function, we are returning nothing here.

  5. https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L155 Are you sure you want to use run.getLogText() this will return annotated text with html. It is meant for use in the UI. You might want to use run.getLog which already returns an array with lines and has annotations removed.

    Actually we did not want to use getLogText(), but getLog(int n) always asks for number of lines as input which we are not sure how to get the total lines. Is there a way to know and pass number of lines to that function or a way to tell just give all lines generated ?

  6. The plugin seems to be focused on git especially github. There are many other SCMs around like gerrit, gitlab, bitbucket, perforce and many more

    Yes this was intentional because on CloudAEye we right now support only github but not other providers

Regards, Bhavya

mawinter69 commented 1 month ago
  1. https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L99 is there a missing return statement?

but you don't want to execute anything afterwards so if you don't add a return statement the rest of the method will still be executed so it should look like this I think

        if(!this.getEnableExport()){
            LOGGER.info(MessageFormat.format("[#{0}] Exporting to CloudAEye is not enabled. Skipping export", run.getNumber()));
            return;
        }
  1. https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L155 Are you sure you want to use run.getLogText() this will return annotated text with html. It is meant for use in the UI. You might want to use run.getLog which already returns an array with lines and has annotations removed.

    Actually we did not want to use getLogText(), but getLog(int n) always asks for number of lines as input which we are not sure how to get the total lines. Is there a way to know and pass number of lines to that function or a way to tell just give all lines generated ?

You could use run.getLog(Integer.MAX_VALUE), so you get the complete log, if there are less lines than you requested it will not fail. Beware that logs can be huge (I have builds that produce logs that are several hundred MiB). You might want to make this configurable, I could imagine that for build error analysis you usually don't need the complete log but only the last few lines.

bhavyalatha26 commented 1 month ago
  1. https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L99 is there a missing return statement?

but you don't want to execute anything afterwards so if you don't add a return statement the rest of the method will still be executed so it should look like this I think

        if(!this.getEnableExport()){
            LOGGER.info(MessageFormat.format("[#{0}] Exporting to CloudAEye is not enabled. Skipping export", run.getNumber()));
            return;
        }
5. > https://github.com/CloudAEye/cloudaeye-plugin/blob/a26b6638f8375a81ab4a6d12dbe46b77c302f92b/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java#L155 Are you sure you want to use run.getLogText() this will return annotated text with html. It is meant for use in the UI. You might want to use run.getLog which already returns an array with lines and has annotations removed.

Actually we did not want to use getLogText(), but getLog(int n) always asks for number of lines as input which we are not sure how to get the total lines. Is there a way to know and pass number of lines to that function or a way to tell just give all lines generated ?

You could use run.getLog(Integer.MAX_VALUE), so you get the complete log, if there are less lines than you requested it will not fail. Beware that logs can be huge (I have builds that produce logs that are several hundred MiB). You might want to make this configurable, I could imagine that for build error analysis you usually don't need the complete log but only the last few lines.

Yes thanks for pointing it out, I also was thinking in same lines that the logs can be huge, is there a way to identify the failed step and only extract logs of that failed step ? I could achieve this with github workflows and AWS codebuild, not sure how to do it with jenkins

bhavyalatha26 commented 1 month ago

New updates:

bhavyalatha26 commented 1 month ago

@mawinter69 Can you please review the updated code and let us know any changes required

Wadeck commented 1 month ago

(sorry for the noise of the bot)

github-actions[bot] commented 1 month ago

Hello from your friendly Jenkins Hosting Checker

It looks like you have everything in order for your hosting request. A member of the Jenkins hosting team will check over things that I am not able to check(code review, README content, etc) and process the request as quickly as possible. Thank you for your patience.

Hosting team members can host this request with /hosting host

bhavyalatha26 commented 1 month ago

/hosting host

bhavyalatha26 commented 1 month ago

@mawinter69 Can you please review the updated code and let us know any changes required

mawinter69 commented 1 month ago
bhavyalatha26 commented 1 month ago
<dependency>
    <groupId>io.jenkins.plugins</groupId>
    <artifactId>gson-api</artifactId>
</dependency>

and

<dependency>
    <groupId>org.jenkins-ci.plugins</groupId>
    <artifactId>apache-httpcomponents-client-4-api</artifactId>
</dependency>
* tenantKey and token in https://github.com/CloudAEye/cloudaeye-plugin/blob/99e376ba9454e3004a5ab432a8837b46ac720581/src/main/java/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications.java, should not be fields in the class. Your intention is to use the value from the global config as you initialize them in the constructor from the global config. Consider that for freestyle jobs the constructor is called when the job is saved after configuring it. So any changes done in the global config afterwards will never arrive there.

* Please do not log with Level `INFO`. This only spams the global Jenkins log. Log with level `FINE`. If you need to see the logs you can always configure a logger via the Jenkins UI and then you just see the logs of your plugin. Instead in some places you should probably use the TaskListener that you get as parameter in the `perform` method, e.g. in the method `sendDetailsToCloudAEye` I think it is important that it is written to the build log if sending was successful or not.

@mawinter69 made all the recommended changes, please review

bhavyalatha26 commented 3 weeks ago

@mawinter69 can you please review the code changes and let us know if anything else is required

Wadeck commented 2 weeks ago

FTR from security perspective everything is fine, it can move forward :)

timja commented 2 weeks ago

@Wadeck does this look ok to you? https://github.com/CloudAEye/cloudaeye-plugin/blob/main/src/main/java/io/jenkins/plugins/cloudaeye/GlobalKeyConfiguration.java#L88

timja commented 2 weeks ago
  1. Please name your class after your plugin key configuration is too generic: https://github.com/CloudAEye/cloudaeye-plugin/blob/main/src/main/java/io/jenkins/plugins/cloudaeye/GlobalKeyConfiguration.java#L25

It will appear in JCasC configuration as globalKeyConfiguration which isn't related to your plugin. e.g. CloudAEyeGlobalConfiguration

  1. Remove this commented out code: https://github.com/CloudAEye/cloudaeye-plugin/blob/main/src/main/resources/io/jenkins/plugins/cloudaeye/GlobalKeyConfiguration/config.jelly#L11-L36

  2. The help files in this folder are unused: https://github.com/CloudAEye/cloudaeye-plugin/tree/main/src/main/resources/io/jenkins/plugins/cloudaeye/CloudAEyeNotifications

Wadeck commented 2 weeks ago

Good catch @timja. @bhavyalatha26 could you ensure to put a permission check on that method? The suppress warning you wrote is incorrect due to the side effect present in the method (it's a SSRF vulnerability)

bhavyalatha26 commented 2 weeks ago

@Wadeck @timja made all the changes recommended in above comments. Please review and let us know if any more cahnges required

timja commented 2 weeks ago

/hosting host

jenkins-infra-bot commented 2 weeks ago

Hosting request complete, the code has been forked into the jenkinsci project on GitHub as https://github.com/jenkinsci/cloudaeye-plugin

GitHub issues has been selected for issue tracking and was enabled for the forked repo.

A pull request has been created against the repository permissions updater to setup release permissions. Additional users can be added by modifying the created file.

Please delete your original repository (if there are no other forks), under 'Danger Zone', so that the jenkinsci organization repository is the definitive source for the code. If there are other forks, please contact GitHub support to make the jenkinsci repo the root of the fork network (mention that Jenkins approval was given in support request 569994). Also, please make sure you properly follow the documentation on documenting your plugin so that your plugin is correctly documented.

You will also need to do the following in order to push changes and release your plugin:

Welcome aboard!