OpenTDF

[!NOTE] It is advised to familiarize yourself with the terms and concepts used in the OpenTDF platform.

Documentation

• Configuration
• Development
• Policy Config Schema
• Policy Config Testing Diagram

Prerequisites for Project Consumers & Contributors

• Go (see go.mod for specific version)
• Container runtime
  • Docker
  • Podman
• Compose - used to manage multi-container applications
  • Docker Compose
  • Podman Compose
• Buf is used for managing protobuf files. Required for developing services.

On macOS, these can be installed with brew

brew install buf go

Optional tools

• Optional Air is used for hot-reload development
  • install with go install github.com/cosmtrek/air@latest
• Optional golangci-lint is used for ensuring good coding practices
  • install with brew install golangci-lint
• Optional grpcurl is used for testing gRPC services
  • install with brew install grpcurl
• Optional openssl is used for generating certificates
  • install with brew install openssl

Audience

There are two primary audiences for this project. Consumers and Contributors

1. Consuming Consumers of the OpenTDF platform should begin their journey here.

2. Contributing To contribute to the OpenTDF platform, you'll need bit more set setup and should start here.

Additional info for Project Consumers & Contributors

For Consumers

The OpenTDF service is the main entry point for the OpenTDF platform. See service documentation for more information.

Quick Start

[!WARNING] This quickstart guide is intended for development and testing purposes only. The OpenTDF platform team does not provide recommendations for production deployments.

To get started with the OpenTDF platform make sure you are running the same Go version found in the go.mod file.

https://github.com/opentdf/platform/blob/main/service/go.mod#L3

Generate development keys/certs for the platform infrastructure.

./.github/scripts/init-temp-keys.sh

Start the required infrastructure with compose-spec.

# Note this might be `podman compose` on some systems
docker compose -f docker-compose.yaml up

Copy the development configuration file from the example and update it with your own values (if necessary, not common).

cp opentdf-dev.yaml opentdf.yaml

Provision keycloak with the default configuration.

go run ./service provision keycloak

Run the OpenTDF platform service.

go run ./service start

For Contributors

This section is focused on the development of the OpenTDF platform.

Libraries

Libraries ./lib are shared libraries that are used across the OpenTDF platform. These libraries are used to provide common functionality between the various sub-modules of the platform monorepo. Specifically, these libraries are shared between the services and the SDKs.

Services

Services ./services are the core building blocks of the OpenTDF platform. Generally, each service is one or more gRPC services that are scoped to a namespace. The essence of the service is that it takes a modular binary architecture approach enabling multiple deployment models.

SDKs

SDKs ./sdk are the contracts which the platform uses to ensure that developers and services can interact with the platform. The SDKs contain a native Go SDK and generated Go service SDKs. A full list of SDKs can be found at github.com/opentdf.

How To Add a New Go Module

Within this repo, todefine a new, distinct go module, for example to provide shared functionality between several existing modules, or to define new and unique functionality follow these steps. For this example, we will call our new module lib/foo.

mkdir -p lib/foo
cd lib/foo
go mod init github.com/opentdf/platform/lib/foo
go work use .

In this folder, create your go code as usual.

Add a README.md and a LICENSE File

A README is recommended to assist with orientation to use of your package. Remember, this will be published to https://pkg.go.dev/ as part of the module documentation.

Make sure to add a LICENSE file to your module to support automated license checks. Feel free to copy the existing (BSD-clear) LICENSE file for most new modules.

Updating the Makefile

1. Add your module to the MODS variable:

   MODS=protocol/go sdk . examples lib/foo

2. If required If your project does not generate a built artifact, add a phony binary target to the .PHONY declaration.

   .PHONY: ...existing phony targets... lib/foo/foo

3. Add your build target to the build phony target.

   build: ...existing targets... lib/foo/foo

4. Add your build target and rule

   lib/foo/foo: $(shell find lib/foo)
   (cd lib/foo && go build ./...)

Updating the Docker Images

Add any required COPY directives to ./Dockerfile:

COPY lib/foo/ lib/foo/

Updating the Workflow Files

1. Add your new go.mod directory to the .github/workflows/checks.yaml's go job's matrix.strategry.directory line.
2. Add the module to the license job in the checks workflow as well, especially if you declare any dependencies.
3. Do the same for any other workflows that should be running on your folder, such as vuln-check and lint.

Terms and Concepts

Common terms used in the OpenTDF platform.

Service is the core service of the OpenTDF platform as well as the sub-services that make up the platform. The main service follows a modular binary architecture, while the sub-services are gRPC services with HTTP gateways.

Policy is the set of rules that govern access to the platform.

OIDC is the OpenID Connect protocol used solely for authentication within the OpenTDF platform.

• IdP - Identity Provider. This is the service that authenticates the user.
• Keycloak is the turn-key OIDC provider used within the platform for proof-of-value, but should be replaced with a production-grade OIDC provider or deployment.

Attribute Based Access Control (ABAC) is the policy-based access control model used within the OpenTDF platform.

• PEP - A Policy Enforcement Point. This is a service that enforces access control policies.
• PDP - A Policy Decision Point. This is a service that makes access control decisions.

Entities are the main actors within the OpenTDF platform. These include people and systems.

• Person Entity (PE) - A person entity is a person that is interacting with the platform.
• Non Person Entity (NPE) - A non-person entity is a service or system that is interacting with the platform.

SDKs are the contracts which the platform uses to ensure that developers and services can interact with the platform.

• SDK - The native Go OpenTDF SDK (other languages are outside the platform repo).
  • A full list of SDKs can be found at github.com/opentdf.
• Service SDK - The SDK generated from the service proto definitions.
  • The proto definitions are maintained by each service.