The Twelve-Factor App is a methodology for building software-as-a-service applications. It aims to optimize portability, scalability, and maintainability of applications by adhering to twelve principles.
The Codebase factor in the Twelve-Factor App methodology proposes that each project or service should have exactly one codebase. This codebase is tracked in version control and can be deployed to multiple environments.
NOTE Separate codebases for different services or splitting a codebase into different repositories does not adhere to this factor. It's about having a single, version-controlled codebase that can be used to deploy to various environments.
Components and Features
Single Codebase
A codebase is the source code for a service or application. In the Twelve-Factor methodology, there should be exactly one codebase per application. If there are multiple codebases, it's a distributed system, each component of which is a twelve-factor app.
Multiple Deployments
The same codebase can be used to deploy to various environments, e.g. production, staging, and development. Each deployment will have a different version of the app running. The codebase is the single source of truth for the app.
Version Control
The codebase for an app is tracked in a version control system, such as Git, and many deploys (dev, stage, production environments) can be executed from the same codebase.
1.2. Dependencies
The Dependencies factor of the Twelve-Factor App methodology is a principle of software engineering that relates to the management of software dependencies by explicitly declaring and isolating dependency.
NOTE Adhering the Dependencies principles improves the portability, scalability, and maintainability of the software. It makes it easier to set up the development environment, to manage the application across different environments, e.g. development, testing, production, and to scale the application by deploying it on multiple servers.
Components and Features
Explicit Declaration
The application should declare all dependencies, using a dependency management tool like pip for Python (requirements.txt), npm for Node.js (package.json).
Isolation
Dependencies should be isolated to prevent conflicts between dependencies of different applications. This can be achieved through virtual environments in Python (venv), or containers (e.g. Docker, Podman).
Examples and Explanations
Dependencies are explicitly declared, and isolation is achieved by installing dependencies in environments that are separate from the system's global environment. This helps to avoid version conflicts between projects.
Python
Python uses pip as its de facto standard package manager. Declare dependencies in a requirements.txt file, which can be generated using pip freeze > requirements.txt.
flask==1.1.2
numpy==1.19.2
pandas==1.1.3
Install dependencies in an isolated environment using a virtual environment.
Node.js uses npm or yarn to manage dependencies, which are declared in a package.json file. Install dependencies using npm install or yarn command in an isolated node_modules directory environment.
Java applications often use Maven for dependency management. Dependencies are declared in a pom.xml file. Maven automatically handles the installation of dependencies in an isolated local Maven repository environment.
The Config factor of the Twelve-Factor App methodology advocates for the strict separation of configuration from code.
NOTE Adhering the Config principle leads to applications that are easy to configure and deploy across different environments, improving developer productivity and system robustness.
Components and Features
Configuration
Any aspect of the application that can vary between deployments such as staging, production, developer environments, should be extracted as configuration. This can include resource handles to the database or other backing services, credentials for external services, and per-deploy values like the canonical hostname for the deploy.
Environment Variables
The Twelve-Factor app encourage to store configuration in environment variables. Env vars interchangeable between deploys without changing any code. Unlike config files, there is a likelihood of being checked into the code repository accidentally and unlike custom config files, or config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.
Examples and Explanations
Environment Variables
Store and access configurations in environment variables using .env files.
The Backing Services factor in the Twelve-Factor App methodology treats all services the application relies on as attached resources. Backing services refers to the treatment of services such as databases, messaging/queueing systems, SMTP/pop3 services, and caching systems.
NOTE Adhering the Backing Services principle, applications achieve flexibility, portability across different environments, and easier scaling when the load increases.
Components and Features
Backing Services
Backing services are any services that an application communicates with over a network. Examples of backing services include databases (like MySQL or MongoDB), messaging/queueing systems (like RabbitMQ or SQS), SMTP services for outgoing email, and caching systems (like Memcache or Redis).
Attached Resources
Treat backing services like databases, message brokers, and caches as attached resources. They should be accessed via a URL or connection string stored in the environment.
Swappable
The backing service should be easily replaceable without changing the application code, enhancing flexibility and scalability.
Examples and Explanations
The backing service (a database, in this case) is treated as an attached resource, and the same code is used to interact with it whether it's a local database or a third-party one.
Python
Python with SQLAlchemy for database interaction. The database URL, whether it's a local database or a third-party one, can be stored as an environment variable.
import os
from sqlalchemy import create_engine
DATABASE_URL = os.getenv('DATABASE_URL')
engine = create_engine(DATABASE_URL)
Node.js
Node.js with Mongoose for MongoDB interaction and object modeling. The MongoDB URI can be stored in an environment variable.
Build, Release, Run emphasizes the need for distinct and well-defined stages in an applications lifecycle.
NOTE The stages should be strictly separated to ensure the reliability, reproducibility, and consistency of application deployments. Any change to the application requires a progression back through these stages, creating a new release and then moving that release to the run stage. This workflow enables version tracking, problem diagnosis, and rollback capabilities if a problem is detected in the running application.
Components and Features
Build Stage
Converts the code repository into an executable bundle. This might involve compiling code, packaging dependencies.
Release Stage
Combines the build with configuration data, resulting in a release that can be deployed to any environment.
Run Stage
Launches the application using a specific release. This stage is responsible for executing the code in the chosen environment.
1.6. Processes
The Processes factor in the Twelve-Factor App methodology emphasizes that applications should be stateless processes and share-nothing. Any data that needs to persist must be stored in a stateful backing service, typically a database.
NOTE Adhering the Processes principle, applications become more scalable and robust, as processes can be replicated (for handling more load) and are resilient to crashes (a crash only affects a single request).
Components and Features
Stateless Processes
Application processes should be stateless and share-nothing. Data that needs to persist should be stored in a stateful backing service such as a database.
Ephemeral
Processes can start and stop at any time. They should be designed to handle this volatility gracefully.
Examples and Explanations
Python
Python application using Flask and a database as backing service for persistent data to avoid storing the state in the application.
from flask import Flask, request
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql://user:pass@localhost/db'
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50))
@app.route('/users', methods=['POST'])
def create_user():
name = request.json['name']
new_user = User(name=name)
db.session.add(new_user)
db.session.commit()
return f'User {name} has been created!', 201
Node.js
Node.js application using Express and a database like MongoDB to avoid storing state in the application.
const express = require('express');
const app = express();
const mongoose = require('mongoose');
const { User } = require('./models/User');
mongoose.connect('mongodb://localhost/test', {useNewUrlParser: true, useUnifiedTopology: true});
app.use(express.json());
app.post('/users', async (req, res) => {
const { name } = req.body;
const user = new User({ name });
await user.save();
res.status(201).send(`User ${name} has been created!`);
});
app.listen(3000);
1.7. Port Binding
The Port Binding factor in the Twelve-Factor App methodology emphasizes that applications should be self-contained and make services available to the outside world by binding to a specified port.
Port binding aligns with the principles of the Twelve-Factor App methodology, promoting portability, flexibility, and scalability. The application can run independently in a variety of environments, making it easier to manage and deploy.
Components and Features
Self-Contained Services
The application should be self-contained and expose services by binding to a port. For web applications, this means running an HTTP server inside the application (e.g., using express in Node.js or Gunicorn in Python).
Port Binding
The application should be configured to bind to a specific port and handle incoming requests.
Environment-Based Configuration
The specific port number that the application binds to might be provided as an environment variable. This allows the configuration to change between different environments, such as development, testing, and production environments. It provides flexibility and helps to keep the environments isolated from each other.
Examples and Explanations
Node.js
Node.js application using Express, setting port number within an environment variable. If that variable isn't set, it defaults to 3000.
const express = require('express');
const app = express();
const port = process.env.PORT || 3000;
app.get('/', (req, res) => res.send('Hello World!'));
app.listen(port, () => console.log(`App listening on port ${port}!`));
Python
Python application using Flask, setting port number within an environment variable. If that variable isn't set, it defaults to 5000.
from flask import Flask
import os
app = Flask(__name__)
port = int(os.getenv('PORT', 5000))
@app.route('/')
def hello_world():
return 'Hello, World!'
if __name__ == '__main__':
app.run(host='0.0.0.0', port=port)
Java
Java application using Spring Boot, setting port number within an environment variable in application.properties file. If that variable isn't set, it defaults to 8080.
# application.properties
server.port=${PORT:8080}
1.8. Concurrency
Concurrency is advocated by scaling out via the process model.
NOTE Adhering the Concurrency principle, applications can effectively handle varying loads, increase their fault tolerance, and improve their overall performance and efficiency.
Components and Features
Process Model
Scale out the application by running multiple processes or instances of the application. This can be achieved by leveraging process management tools or orchestrators like Kubernetes.
Types of Processes
Different types of processes (web servers, background workers) can be used to handle different workloads and improve concurrency. For instance, long-running tasks might be handled by a worker process, while short, request/response tasks might be handled by a web process.
Scaling Out
The Twelve-Factor App methodology emphasizes scaling out (horizontal scaling), rather than scaling up (vertical scaling). Scaling out means increasing the number of processes to handle more tasks simultaneously. This allows the application to distribute the load across multiple processes, making it more resilient and adaptable to changes in load. It contrasts with scaling up, which involves increasing the computational resources of an individual component.
Examples and Explanations
Node.js
Node.js has a built-in module called Cluster to create child processes (workers) that share server ports to scale an application.
const cluster = require('cluster');
const os = require('os');
if (cluster.isMaster) {
const cpuCount = os.cpus().length; // Get the number of CPUs
// Create a worker for each CPU
for (let i = 0; i < cpuCount; i++) {
cluster.fork();
}
} else {
const express = require('express');
const app = express();
app.get('/', (req, res) => res.send('Hello from Worker!'));
app.listen(3000);
}
Java
Java using Spring Boot and Executors framework to manage threads to work with many tasks concurrently.
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class App {
private static final int NTHREDS = 10;
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(NTHREDS);
for (int i = 0; i < 500; i++) {
Runnable worker = new MyRunnable(10000000L + i);
executor.execute(worker);
}
executor.shutdown();
while (!executor.isTerminated()) {
}
System.out.println("Finished all threads");
}
}
1.9. Disposability
The Disposability factor in the Twelve-Factor App methodology emphasizes that applications should strive for maximum robustness with fast startup and graceful shutdown.
Applications that adhere to the disposability factor improve their robustness and resilience. They can handle unexpected changes in system state, such as sudden increases in load or crashes. They are also more amenable to rapid scaling, as new instances can be started quickly.
Components and Features
Fast Startup
Applications should start quickly to facilitate rapid deployment and scaling.
Graceful Shutdown
Applications should shut down gracefully to handle termination signals (SIGTERM) properly, allowing for tasks to be completed or cleaned up.
Examples and Explanations
Node.js
Node.js using Express to listen for the SIGTERM signal and shut down gracefully.
Python applications using Flask to listen for the SIGTERM signal, designed to shut down gracefully when it receives a signal.
from flask import Flask
import os
import signal
import sys
app = Flask(__name__)
def graceful_shutdown(signal, frame):
print('SIGTERM received. Shutting down gracefully.')
sys.exit()
signal.signal(signal.SIGTERM, graceful_shutdown)
if __name__ == '__main__':
app.run(host='0.0.0.0', port=int(os.getenv('PORT', 5000)))
Java
Java application using Spring Boot @PreDestroy annotation to do cleanup before the application is terminated.
import javax.annotation.PreDestroy;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
@PreDestroy
public void preDestroy() {
System.out.println("Application is about to terminate...");
// Perform cleanup here
}
}
1.10. Dev/Prod Parity
The Dev/Prod Parity factor in the Twelve-Factor App methodology emphasizes that the gap between development and production should be kept as small as possible.
NOTE Minimizing the gaps, increases the consistency between environments, reduce unexpected bugs or issues during deployment, and improve the reliability of the application.
Components and Features
Time Gap
Minimize the time gap between writing code and deploying it to production, e.g. Continuous Integration/Continuous Deployment (CI/CD) pipelines.
Identical Environments
Keep development, staging, and production environments as similar as possible. Use the same type of backing services, same configurations, and similar infrastructure.
Examples and Explanations
Environment Variables
Environment variables are a great way to manage settings that vary between environments.
The database connection string might be different in development and production. The MONGODB_URI environment variable holds the connection string. In development, it's set to connect to a local MongoDB instance. In production, it's set connect to a MongoDB Atlas cluster.
const mongoose = require('mongoose');
const uri = process.env.MONGODB_URI;
mongoose.connect(uri);
Docker
Dockerfile for a Node.js application using Docker to ensure that the application runs in the same way in different environments.
The Dockerfile describes the workflow to build a Docker image of the application. Run the image in any environment that has Docker installed, ensuring that the application's runtime environment is consistent across different stages.
# Use an official Node.js runtime as the base image
FROM node:14
# Set the working directory in the container
WORKDIR /usr/src/app
# Copy package.json and package-lock.json into the container
COPY package*.json ./
# Install the application's dependencies inside the container
RUN npm install
# Copy the rest of the application's code into the container
COPY . .
# Make port 8080 available outside the container
EXPOSE 8080
# Start the application
CMD [ "node", "server.js" ]
1.11. Logs
The Logs factor in the Twelve-Factor App methodology emphasizes that applications should treat logs as event streams.
Application should not be concerned with routing and storage of its output stream. Instead, each running process writes its event stream, unbuffered, to stdout. Making it easy to collect logs with a log aggregation tool, such as Fluentd, Grafana Loki, or Logstash, and then send them to a centralized log management service for analysis.
NOTE Treating logs as event streams and decoupling log management from the application, create systems that are easier to observe, debug, and maintain.
Components and Features
Event Streams
Treat logs as continuous event streams. Write logs to stdout (standard output) and let the environment handle the aggregation, storage, and analysis.
Log Routing
Logs should be captured by the execution environment, collated together with logs from other applications, and then forwarded to a centralized log indexing and analysis system. This allows for more sophisticated analysis and troubleshooting across multiple services. Use tools like Logstash, Fluentd, or cloud-based logging services to collect and analyze logs.
Examples and Explanations
Node.js
Node.js, using Express with console.log() to write log messages to stdout.
const express = require('express');
const app = express();
app.get('/', (req, res) => {
console.log('GET request received at /');
res.send('Hello World!');
});
app.listen(3000, () => console.log('App listening on port 3000!'));
Python
Python using Flask with logging module to write log messages to stdout.
from flask import Flask
import logging
app = Flask(__name__)
@app.route('/')
def hello_world():
app.logger.info('GET request received at /')
return 'Hello, World!'
if __name__ == '__main__':
logging.basicConfig(level=logging.INFO)
app.run(host='0.0.0.0', port=5000)
Java
Java using Spring Boot with Logback or Log4j to write log messages to stdout.
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
@SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
@RestController
public class HelloController {
Logger logger = LoggerFactory.getLogger(HelloController.class);
@GetMapping("/")
public String hello() {
logger.info("GET request received at /");
return "Hello, World!";
}
}
}
1.12. Admin Processes
The Admin Processes factor in the Twelve-Factor App methodology emphasizes that one-off administrative or management tasks should be run in an identical environment as the regular long-running processes of the app.
The one-off admin processes run in the same environment as the app and are managed as part of the same codebase. This ensures that they're working with the same code and following the same release lifecycle.
NOTE Adhering the Admin Processes principle, ensures that all tasks, regular and administrative, are subject to the same environmental conditions, preventing discrepancies and potential issues.
Components and Features
One-Off Tasks
Administrative tasks such as database migrations, maintenance scripts, or debugging tasks should be run as one-off processes in an environment identical to the application’s runtime.
Environment Consistency
Ensure that admin tasks have the same dependencies and configuration as the main application processes. This prevents issues caused by environmental differences and ensures consistency across all operations of the app.
Part of Application Codebase
Administrative code should be included in the application's codebase. This way, it evolves with the rest of the application, ensuring that these tasks can always be executed without compatibility issues.
Examples and Explanations
Node.js
In Node.js might have a script to seed a database. This script would be run as a one-off process but within the same environment. This could be a script in package.json. Run the seed script with npm run seed.
In Python using Django, create a custom management commands. These can be run from the command line and are perfect for administrative tasks. Run the seed command with python manage.py seed.
# app/management/commands/seed.py
from django.core.management.base import BaseCommand
from app.models import MyModel
class Command(BaseCommand):
help = 'Seeds the database'
def handle(self, *args, **options):
MyModel.objects.create(...)
Java
In Java using Spring Boot, create a CommandLineRunner bean for tasks that should be executed after the application context is loaded. Conditionally run the seeder based on a program argument or an environment variable.
import org.springframework.boot.CommandLineRunner;
import org.springframework.stereotype.Component;
@Component
public class Seeder implements CommandLineRunner {
@Override
public void run(String... args) throws Exception {
// Seed database here
}
}
Twelve-Factor App
The Twelve-Factor App is a methodology for building software-as-a-service applications. It aims to optimize portability, scalability, and maintainability of applications by adhering to twelve principles.
1. Category
1.1. Codebase
The
Codebasefactor in the Twelve-Factor App methodology proposes that each project or service should have exactly one codebase. This codebase is tracked in version control and can be deployed to multiple environments.Components and Features
Single Codebase
Multiple Deployments
Version Control
1.2. Dependencies
The
Dependenciesfactor of the Twelve-Factor App methodology is a principle of software engineering that relates to the management of software dependencies by explicitly declaring and isolating dependency.Components and Features
Explicit Declaration
Isolation
Examples and Explanations
Dependencies are explicitly declared, and isolation is achieved by installing dependencies in environments that are separate from the system's global environment. This helps to avoid version conflicts between projects.
Python
Install dependencies in an isolated environment using a virtual environment.
Node.js
Java
1.3. Config
The
Configfactor of the Twelve-Factor App methodology advocates for the strict separation of configuration from code.Components and Features
Configuration
Environment Variables
Examples and Explanations
Environment Variables
Python
Node.js
Java
1.4. Backing Services
The
Backing Servicesfactor in the Twelve-Factor App methodology treats all services the application relies on as attached resources. Backing services refers to the treatment of services such as databases, messaging/queueing systems, SMTP/pop3 services, and caching systems.Components and Features
Backing Services
Attached Resources
Swappable
Examples and Explanations
The backing service (a database, in this case) is treated as an attached resource, and the same code is used to interact with it whether it's a local database or a third-party one.
Python
Node.js
Java
1.5. Build, Release, Run
Build, Release, Runemphasizes the need for distinct and well-defined stages in an applications lifecycle.Components and Features
Build Stage
Release Stage
Run Stage
1.6. Processes
The
Processesfactor in the Twelve-Factor App methodology emphasizes that applications should be stateless processes and share-nothing. Any data that needs to persist must be stored in a stateful backing service, typically a database.Components and Features
Stateless Processes
Ephemeral
Examples and Explanations
Python
Node.js
1.7. Port Binding
The
Port Bindingfactor in the Twelve-Factor App methodology emphasizes that applications should be self-contained and make services available to the outside world by binding to a specified port.Port binding aligns with the principles of the Twelve-Factor App methodology, promoting portability, flexibility, and scalability. The application can run independently in a variety of environments, making it easier to manage and deploy.
Components and Features
Self-Contained Services
Port Binding
Environment-Based Configuration
Examples and Explanations
Node.js
Python
Java
1.8. Concurrency
Concurrency is advocated by scaling out via the process model.
Components and Features
Process Model
Types of Processes
Scaling Out
Examples and Explanations
Node.js
Java
1.9. Disposability
The
Disposabilityfactor in the Twelve-Factor App methodology emphasizes that applications should strive for maximum robustness with fast startup and graceful shutdown.Applications that adhere to the disposability factor improve their robustness and resilience. They can handle unexpected changes in system state, such as sudden increases in load or crashes. They are also more amenable to rapid scaling, as new instances can be started quickly.
Components and Features
Fast Startup
Graceful Shutdown
Examples and Explanations
Node.js
Python
Java
1.10. Dev/Prod Parity
The
Dev/Prod Parityfactor in the Twelve-Factor App methodology emphasizes that the gap between development and production should be kept as small as possible.Components and Features
Time Gap
Identical Environments
Examples and Explanations
Environment Variables
Docker
1.11. Logs
The
Logsfactor in the Twelve-Factor App methodology emphasizes that applications should treat logs as event streams.Application should not be concerned with routing and storage of its output stream. Instead, each running process writes its event stream, unbuffered, to stdout. Making it easy to collect logs with a log aggregation tool, such as
Fluentd,Grafana Loki, orLogstash, and then send them to a centralized log management service for analysis.Components and Features
Event Streams
Log Routing
Examples and Explanations
Node.js
Python
Java
1.12. Admin Processes
The
Admin Processesfactor in the Twelve-Factor App methodology emphasizes that one-off administrative or management tasks should be run in an identical environment as the regular long-running processes of the app.The one-off admin processes run in the same environment as the app and are managed as part of the same codebase. This ensures that they're working with the same code and following the same release lifecycle.
Components and Features
One-Off Tasks
Environment Consistency
Part of Application Codebase
Examples and Explanations
Node.js
Python
Java
2. References