Add support for Jenkins

[maxrake]

Overview

The phylum Python package should be updated to offer the ability to analyze dependency file changes within the Jenkins CI environment. Specifically, the Jenkins Pipeline should be supported and documented for use with declarative pipeline syntax in multibranch pipelines. Pipeline runs for both PRs and branches should be supported.

Additional context

Unsorted Links and References

Jenkins user documentation Pipeline syntax Using credentials Using Docker with Pipeline Pipeline steps reference Supporting pull requests Jenkins environment variables Global variable reference

• Default variables - found in local instance (e.g., http://localhost:8080/pipeline-syntax/globals#env)
  • from the watson/ci-info project
  • Detecting when in a build: JENKINS_URL and BUILD_ID
  • Detecting when in a PR: ghprbPullId or CHANGE_ID
  • BRANCH_NAME: For a multibranch project, this will be set to the name of the branch being built, for example in case you wish to deploy to production from master but not from feature branches; if corresponding to some kind of change request, the name is generally arbitrary (refer to CHANGE_ID and CHANGE_TARGET).
  • BRANCH_IS_PRIMARY: For a multibranch project, if the SCM source reports that the branch being built is a primary branch, this will be set to "true"; else unset. Some SCM sources may report more than one branch as a primary branch while others may not supply this information.
  • CHANGE_ID: For a multibranch project corresponding to some kind of change request, this will be set to the change ID, such as a pull request number, if supported; else unset.
  • CHANGE_TARGET: For a multibranch project corresponding to some kind of change request, this will be set to the target or base branch to which the change could be merged, if supported; else unset.
  • CHANGE_BRANCH: For a multibranch project corresponding to some kind of change request, this will be set to the name of the actual head on the source control system which may or may not be different from BRANCH_NAME. For example in GitHub or Bitbucket this would have the name of the origin branch whereas BRANCH_NAME would be something like PR-24.
  • BUILD_NUMBER: The current build number, such as "153".
  • BUILD_ID: The current build ID, identical to BUILD_NUMBER for builds created in 1.597+, but a YYYY-MM-DD_hh-mm-ss timestamp for older builds.
  • BUILD_DISPLAY_NAME: The display name of the current build, which is something like #153 by default.

Questions for implementation

• Is it possible to run a job in the pipeline from a Docker image?
  • Yes. You can define images at the pipeline or stage level.
• Is it possible to get all the info needed from git within a pipeline/job run?
  • Is git available?
  • Yes, from the Docker image
  • Are shallow clones the default?
  • Yes
  • Is it possible to configure for deep/full clones/checkouts?
  • Yes. Have to use a checkout step to get full history
  • Are PRs created with a unique "merge commit"?
  • It looks like it
• Which environment variables are needed to create an implementation for this CI environment?
  • See the variables above
• Is it possible to use a PAT that covers a larger scope than just the repository (Repo Access Token)?
  • Jenkins integrates with many different source repositories and the required credentials will be specific to the particular environment (e.g., GitHub, GitLab, etc.)
• Is it possible to define both branch and pull request pipelines, but ensure they don't both run at the same time?
  • Yes, with the Exclude branches that are also filed as PRs strategy of the Discover branches option in the Branch sources section of the Multibranch Pipelines configuration

Possible steps for the implementation

• Install/host Jenkins locally
  • Configure user account settings
• Create a Multibranch Pipelines item
  • Configure settings
• Create a source repository
  • Configure settings
  • Start with GitHub
  • Optionally, confirm operation with other supported repo sources
• Populate the repository with basic content/files, to include a dependency lockfile
• Create a pipeline
  • Configure/populate the initial Jenkinsfile
• Ensure the basic phylum-ci use case runs
  • Do this before adding any support for this CI environment...or maybe just the bare minimum to see it running
• Create an implementation for the "Jenkins" CI environment
  • Iterate on it until the results are as expected
  • Support both plain builds and pull requests

Acceptance criteria

• [x] The phylum package offers the ability to analyze dependency file changes in the Jenkins CI ecosystem
• [x] Jenkins CI integration detects and handles both branch pipelines and pull request pipelines
• [x] Documentation is updated