Improve CICD Pipeline

[kechenkristin]

What feature do you want to see added?

As part of our effort to improve the Jenkins-based pipeline for our Java project, I propose implementing the following enhancements:

Continuous Integration (CI) Steps:

Style Check: Utilize Checkstyle and PMD for code style analysis. Unit Tests: Integrate JUnit or TestNG for unit testing. Integration Tests: Implement integration tests using JUnit or TestNG. Code Coverage: Measure code coverage using JaCoCo or Cobertura. Static Code Analysis: Use FindBugs, SpotBugs, or SonarQube for static code analysis. Dependency Check: Integrate OWASP Dependency-Check for scanning dependencies for security vulnerabilities. Artifact Generation: Build artifacts using Maven or Gradle. Documentation Generation: Generate documentation using Javadoc or Doxygen. Continuous Delivery (CD) Steps:

Artifact Promotion: Automate promotion of artifacts between environments. Environment Provisioning: Use Terraform or Ansible for automatic environment provisioning. Configuration Management: Manage environment-specific configurations using Consul, ZooKeeper, or Spring Cloud Config. Database Migrations: Automate database schema migrations with Flyway or Liquibase. Blue-Green Deployments: Implement blue-green deployments for minimizing downtime. Canary Deployments: Deploy new versions to a subset of users for validation. Rollback Strategy: Define procedures for rolling back deployments in case of failures. Smoke Testing: Perform basic smoke tests in production post-deployment. Health Checks: Monitor deployed services' health with Prometheus or Nagios. Auto Scaling: Utilize cloud-native features or Kubernetes HPA for auto-scaling. Performance Monitoring: Monitor application performance using New Relic, Datadog, or Grafana. Continuous Feedback Loop: Gather feedback from users and stakeholders for future improvements.

See example from: https://github.com/jenkins-infra/jenkins-infra/blob/production/Jenkinsfile

Upstream changes

No response

[dduportal]

Hi @kechenkristin , thanks for opening all these issues and making proposals! Welcome to the Jenkins infrastructure area!

Could you describe the problem you would want to solve in the area of CI/CD please?

I'm asking because I only see a list of technologies and tools but would want to start with the "why", e.g. what is the problem you see today on the CI/CD pipeline of this project (ref. https://github.com/jenkins-infra/backend-extension-indexer/blob/master/Jenkinsfile)

Besides, https://github.com/jenkins-infra/jenkins-infra/blob/production/Jenkinsfile is a distinct repository with different goals which is unrelated to the backend-extension-indexed so it is not an example to apply in the context here.

Regarding the CD part, in particular, (as it is the area of my team for the Jenkins infrastructure), the current process is Continuous Deployment: e.g. any commit which passes the pipeline steps and is able to generate the zip artifact (see "Publish step" - https://github.com/jenkins-infra/backend-extension-indexer/blob/242aa3a5d1dc31e459bde0cfd4b0d32ffffd0a49/Jenkinsfile#L41) is deployed to reports.jenkins.io automatically as per https://github.com/jenkins-infra/backend-extension-indexer/blob/242aa3a5d1dc31e459bde0cfd4b0d32ffffd0a49/Jenkinsfile#L43.

=> I'm not sure why did you mentioned all this stuff about CD as this is only a set of static files?

[kechenkristin]

Hi @kechenkristin , thanks for opening all these issues and making proposals! Welcome to the Jenkins infrastructure area!

Could you describe the problem you would want to solve in the area of CI/CD please?

I'm asking because I only see a list of technologies and tools but would want to start with the "why", e.g. what is the problem you see today on the CI/CD pipeline of this project (ref. https://github.com/jenkins-infra/backend-extension-indexer/blob/master/Jenkinsfile)

Besides, https://github.com/jenkins-infra/jenkins-infra/blob/production/Jenkinsfile is a distinct repository with different goals which is unrelated to the backend-extension-indexed so it is not an example to apply in the context here.

Regarding the CD part, in particular, (as it is the area of my team for the Jenkins infrastructure), the current process is Continuous Deployment: e.g. any commit which passes the pipeline steps and is able to generate the zip artifact (see "Publish step" -

https://github.com/jenkins-infra/backend-extension-indexer/blob/242aa3a5d1dc31e459bde0cfd4b0d32ffffd0a49/Jenkinsfile#L41

) is deployed to reports.jenkins.io automatically as per https://github.com/jenkins-infra/backend-extension-indexer/blob/242aa3a5d1dc31e459bde0cfd4b0d32ffffd0a49/Jenkinsfile#L43

. => I'm not sure why did you mentioned all this stuff about CD as this is only a set of static files?

Hi, thank you for your reply, yes you are correct, I agree with you about keeping the current CD as it is right now. However, I'd like to improve the CI part to enable style checks, unit test checks, integration checks, and code coverage checks. The tool list can be regarded as a reference for potential solutions. In the future time, I am happy to contribute to this part as well.

[dduportal]

The tool list can be regarded as a reference for potential solutions. In the future time, I am happy to contribute to this part as well.

But again, my question remains: what problem do you want to solve. It is NOT a of solution list because it does not explain the problem neither the context.

You might want to start on the CI as you explained but don't forget to be clear about the goals to achieve to avoid spreading your efforts :)

[kechenkristin]

The tool list can be regarded as a reference for potential solutions. In the future time, I am happy to contribute to this part as well.

But again, my question remains: what problem do you want to solve. It is NOT a of solution list because it does not explain the problem neither the context.

You might want to start on the CI as you explained but don't forget to be clear about the goals to achieve to avoid spreading your efforts :)

Thank you for your reply and kind guidance. :) I want to make this project more modernized and follow good practice, so I think it would be better to bring style-check, unit test-check, integration test-check, etc to the scope of this issue.