Projen Pipelines is an open-source project that automates the generation of CI/CD pipelines using Projen, a project configuration tool created by the inventor of AWS CDK. It provides high-level abstractions for defining continuous delivery (CD) pipelines for applications, specifically designed to work with the projen project configuration engine.
While Projen Pipelines currently focuses on AWS CDK applications, our vision extends far beyond this initial scope. We aim to evolve into a universal CI/CD pipeline generator capable of supporting a wide variety of application types and deployment targets.
By broadening our scope, we aim to create a tool that can standardize and simplify CI/CD pipeline creation across the entire spectrum of modern application development and deployment scenarios. We invite the community to join us in this journey, contributing ideas, use cases, and code to help realize this vision.
Under the hood, after you define the pipeline and select the target engine you want to work on, we use code generation methods to create the required CI/CD pipeline in your project.
We are considering allowing the selection of multiple engines going forward - please let us know if this is a feature you would use!
To install the package, add the package projen-pipelines
to your projects devDeps in your projen configuration file.
After installing the package, you can import and use the constructs to define your CDK Pipelines.
You will also have to setup an IAM role that can be used by GitHub Actions. You can find a tutorial on how set this up here: Configuring OpenID Connect in Amazon Web Services
You can start using the constructs provided by Projen Pipelines in your AWS CDK applications. Here's a brief example:
import { awscdk } from 'projen';
import { GithubCDKPipeline } from 'projen-pipelines';
// Define your AWS CDK TypeScript App
const app = new awscdk.AwsCdkTypeScriptApp({
cdkVersion: '2.150.0',
name: 'my-awesome-app',
defaultReleaseBranch: 'main',
devDeps: [
'projen-pipelines',
],
});
// Create the pipeline
new GithubCDKPipeline(app, {
stackPrefix: 'MyApp',
iamRoleArns: {
default: 'arn:aws:iam::123456789012:role/GithubDeploymentRole',
},
pkgNamespace: '@company-assemblies',
useGithubPackagesForAssembly: true,
stages: [
{
name: 'dev',
env: { account: '123456789012', region: 'eu-central-1' },
}, {
name: 'prod',
manualApproval: true,
env: {account: '123456789012', region: 'eu-central-1' },
}],
});
After running projen (npx projen
) a new file called src/app.ts
will be created and contain a specialized CDK App class for your project.
You can then use this in your main.ts
to configure your deployment.
import { PipelineApp } from './app';
import { BackendStack } from './stack';
const app = new PipelineApp({
provideDevStack: (scope, id, props) => {
return new BackendStack(scope, id, {
...props,
apiHostname: 'api-dev',
myConfigSetting: 'value-for-dev',
});
},
provideProdStack: (scope, id, props) => {
return new BackendStack(scope, id, {
...props,
apiHostname: 'api',
myConfigSetting: 'value-for-prod',
});
},
providePersonalStack: (scope, id, props) => {
return new BackendStack(scope, id, {
...props,
apiHostname: `api-${props.stageName}`,
myConfigSetting: 'value-for-personal-stage',
});
},
});
app.synth();
When planning to manage multiple staging environments, you will need to establish trust relationships. This process centralizes deployment control, improving operational efficiency and security by consolidating deployment management through a singular, monitored channel. Here is a simplified diagram for the setup:
Bootstrapping initializes the AWS CDK environment in each account. It prepares the account to work with AWS CDK apps deployed from other accounts. Use the cdk bootstrap
command for this purpose. Replace <deployment_account_id>
with the actual AWS account ID of your deployment account.
You can use the CloudShell to bootstrap each staging account:
cdk bootstrap --trust <deployment_account_id> --cloudformation-execution-policies "arn:aws:iam::aws:policy/AdministratorAccess"
Note:
While AdministratorAccess
grants full access to all AWS services and resources, it's not recommended for production environments due to security risks. Instead, create custom IAM policies that grant only the necessary permissions required for deployment operations.
The <Engine>CDKPipeline
class creates and adds several tasks to the projen project that then can be used in your pipeline to deploy your application to AWS.
Here's a brief description of each one:
deploy:personal - This task deploys the application's personal stage, which is a distinct, isolated deployment of the application. The personal stage is intended for personal use, such as testing and development.
watch:personal - This task deploys the personal stage of the application in watch mode. In this mode, the AWS CDK monitors your application source files for changes, automatically re-synthesizing and deploying when it detects any changes.
diff:personal - This task compares the deployed personal stage with the current state of the application code. It's used to understand what changes would be made if the application were deployed.
destroy:personal - This task destroys the resources created for the personal stage of the application.
deploy:feature - This task deploys the application's feature stage. The feature stage is used for new features testing before these are merged into the main branch.
diff:feature - This task is similar to diff:personal
, but for the feature stage.
destroy:feature - This task destroys the resources created for the feature stage of the application.
deploy:
diff:
publish:assets - This task publishes the CDK assets to all accounts. This is useful when the CDK application uses assets like Docker images or files from the S3 bucket.
bump - This task bumps the version based on the latest git tag and pushes the updated tag to the git repository.
release:push-assembly - This task creates a manifest, bumps the version without creating a git tag, and publishes the cloud assembly to your registry.
Remember that these tasks are created and managed automatically by the CDKPipeline
class. You can run these tasks using the npx projen TASK_NAME
command.
Projen-Pipelines is currently in version 0.x, awaiting Projen's 1.0 release. Despite its pre-1.0 status, it's being used in several production environments.
Use the Github integrated "Issues" view to create an item that you would love to have added to our open source project.
No request is too big or too small - get your thoughts created and we'll get back to you if we have questions!
We welcome all contributions to Projen Pipelines! Here's how you can get started:
Fork the Repository: Click the 'Fork' button at the top right of this page to duplicate this repository in your GitHub account.
Clone your Fork: Clone the forked repository to your local machine.
git clone https://github.com/<your_username>/projen-pipelines.git
git checkout -b my-branch
Make your Changes: Make your changes, additions, or fixes to the codebase. Remember to follow the existing code style.
Test your Changes: Before committing your changes, make sure to test them to ensure they work as expected and do not introduce bugs.
Commit your Changes: Commit your changes with a descriptive commit message using conventional commit messages.
git commit -m "feat: Your descriptive commit message"
git push -u origin my-branch
main
branch of this repository.Your pull request will be reviewed and hopefully merged quickly. Thanks for contributing!
The best way currently is to test things locally or - if you have a working stall of all supported CI/CD tools - manually test the functionalities there in diferent projects.
For local testing:
Using yalc push
you can install the project locally to your local yalc package manager. You can also use npm run local-push
instead of this.
With yalc add projen-pipelines
you can then use it in a local project.
Join us in elevating CI/CD pipeline discussions from implementation details to fundamental building blocks, and help create a more efficient, standardized approach to pipeline development!
npx projen
When attempting to run npx projen
, users may encounter an error related to environment variable substitution within configuration files. Specifically, the ${GITHUB_TOKEN}
placeholder fails to be replaced.
To resolve this issue, prefix the npx projen
command with the GITHUB_TOKEN=
environment variable:
GITHUB_TOKEN= npx projen