Analyze and propose a solution to store all secrets

[ilicstef94]

• write in the comment what the proposed solution is
• organize a meeting with the team members and show them the results of the research

[bojanabogicevic]

Considering that deployment is not in the picture yet, and storing secrets as environmental variables is one of the best practices when it comes to storing secrets locally. I would propose storing them in one file per microservice, where we would have a template file in each microservice containing names of secrets necessary for running the app. With this we would have them grouped and we could easily document them in ReadMe file, also we would avoid constant changes in the run configuration of our chosen IDE.

Example for template:

SECRET_VARIABLE_NAME1= SECRET_VARIABLE_NAME2=

[ilicstef94]

In all cases when it comes to confidential data, it is preferable not to display it in the form of plain text, but to be encrypted, even when we need them for the local environment. Within this analysis, I think the focus should be on finding some mechanism for storing secrets. It is best to list as many ways as possible to do this, and to draw out the advantages and disadvantages of them.

[bogdanmilojevic]

Analyze and propose a solution to store all secrets #9

Introduction

This document provides an in-depth exploration of various methods to store secrets for Spring Boot applications. Each approach is carefully examined, highlighting its respective advantages and disadvantages, accompanied by crucial considerations.

Secrets encompass vital data essential for configuring applications, including tokens, passwords, secret IDs, and other sensitive credentials. The effective and secure storage of these secrets is a critical task that demands careful consideration.

Enterprise solutions such as AWS Key Management Service and Azure Key Vault aren’t included in the analysis due to the scope of the project. In reality, managed cloud-based solutions are usually the best choice but often the most expensive one as well.

1. Environment Variables

Environment variables are a good choice for storing non-sensitive data in a trusted environment but they can become major security risk if not handled carefully. Storing sensitive data in environment variables isn’t a viable approach and it shouldn’t be considered in a production-grade software. This approach introduces multiple points of failure, given that same environment variables are usually shared between machines during development.

Use when: Storing non-sensitive data such as environment specific configuration and for local development. Environment variables can be useful for referencing secrets stored on centralized vaults such as Azure KeyVault etc. which is a common approach

Advantages:

• Very easy to setup and manage
• Doesn’t include additional costs
• Widely supported

Disadvantages:

• Usually stored in plain text, not suitable for storing sensitive data
• Sharing environment variables with other developers may pose a security risk
• Managing environment variables in a team can become difficult

Environment variables can be set using .env files (preferable approach along with .env.template files), IntelliJ IDEA run configurations or trough combination of both.

2. Importing External Configuration

Spring Boot Configuration Properties

In a case there is a need to define custom properties which will be filled in application.properties file, Spring Boot provides @ConfigurationProperties annotation.

Benefit of this approach over simply writing arbitrary properties in configuration files and referencing them in code is the IDE support and auto-completion that becomes available once we define custom properties class. This way we can reduce room for error when dealing with configuration. Main drawback is additional setup needed to make it work.

The @ConfigurationProperties annotation can be used above a class that contains fields representing custom properties that are needed:

@ConfigurationProperties("kafka-topics")
public record KafkaTopics(String notificationTopicName, String accountTopicName) {}

Dependency “org.springframework.boot:spring-boot-configuration-processor” must be present on classpath in order to properly process @ConfigurationProperties annotation.

@EnableConfigurationProperties(KafkaProperties.class) annotation must be added to Application class as well.

After defining a class and building the project, new custom properties can be set in application.properties file:

kafka-topics.notification-topic-name=notification-topic
kafka-topics.account-topic-name=account-topic

Or application.yml:

kafka-topics:
    notification-topic-name: notification-topic
    account-topic-name: account-topic

If we want to use these values in the code we can simply inject KafkaTopics bean and access the fields we need.

Handling Secret Values

Although @ConfigurationProperties represent a handy way to manage custom properties, it doesn’t do much regarding storing secrets. Configuration values will be checked into repository if left inside application.properties/yml which is a bad practice.

In order to bypass this we can add new secrets.properties file, add it to .gitignore and import it in main configuration file like this:

spring.config.import=optional:secret.properties

This approach is pretty similar to using simple environment variables and it poses same security risks so it isn’t the best choice for storing sensitive data, especially when the data is stored in plaintext.

3. Jasypt Encryption

Major security risk associated with previous two methods is that both store secrets in plaintext. Simple and straight-forward way to encrypt secrets is to use [Jasypt](http://www.jasypt.org/) library.

Usage:

secretparam=mysecretvalue123

@Value("${secretparam}")
private String secretParam;

• Add @EnableEncryptableProperties above main application class
• Add command line argument (VM options): -Djasypt.encryptor.password=password This password will be used to decrypt secret properties

Wrap secret value inside ENC()

secretparam=ENC(mysecretvalue123)

Run Jasypt maven plugin:

mvn jasypt:encrypt-value -Djasypt.encryptor.password="password" -Djasypt.plugin.value="mysecretvalue123"

Plugin will output encrypted secret value which now can be set instead of plaintext value:

secretparam=ENC(cf9P5QWJVrpnXQR3L8xwwqIbw0fbNhnRYdds)

Advantages:

• Fast and easy way to encrypt secrets for local development.
• Highly configurable

Disadvantages:

• It seems that Jasypt is no longer maintained, last version was released in May 2019.
• Storing Jasypt password is one more thing that needs consideration.
• Not suitable for production due to lack of maintaince.

4. Spring Vault (with HashiCorp Vault)

Spring Vault is a spring project which provides abstractions over HashiCorp Vault, making it easy to use in Spring applications.

Spring Cloud Vault Config provides client-side support for externalized configuration in a distributed system. With [HashiCorp’s Vault](https://www.vaultproject.io/) you have a central place to manage external secret properties for applications across all environments. Vault can manage static and dynamic secrets such as username/password for remote applications/resources and provide credentials for external services such as MySQL, PostgreSQL, Apache Cassandra, MongoDB, Consul, AWS and more.

HashiCorp Vault is an identity-based secrets and encryption management system. It provides encryption services that are gated by authentication and authorization methods to ensure secure, auditable and restricted access to secrets. It is used to secure, store and protect secrets and other sensitive data using a UI, CLI, or HTTP API.

Advantages:

• Proven and production ready solution.
• Vault validates and authorizes clients (users, machines, apps) before providing them access to secrets or stored sensitive data.
• Vault can generate dynamic secrets
• Data encryption

Disadvantages:

• Spring Cloud Config and HashiCorp Vault servers become two more thing to manage when deployed to production.
• Introduces more complexity.
• Requires a fair amount of initial setup to make everything work.

Proposed Solution for MicroCommerce

• Setting up new Spring Cloud Config server which will act as a centralized place for providing configuration to all services.
• Spring Cloud Configuration server will internally talk to HashiCorp Vault server which will store sensitive data.
• Services will connect to the Spring Cloud Config and pull needed configuration details.

In my opinion, while this method adds more complexity, it stands as the most practical solution applicable to real-life scenarios.