Also see the DataONE Certificate Creation - Quick Reference Guide
This spreadsheet is updated weekly by a GitHub Action
This directory contains configuration files and notes on how to set up the DataONE Certificate Authority using OpenSSL as the CA application. OpenSSL generates all files needed for the CA, including certificate requests, keys, certificates, and certificate revocation lists.
The DataONE Certificate Authority is governed by a Root CA, which delegates
all certificate signing and management to a Production CA. The private key
for the Root CA is offline and completely protected, which protects the CA
should somehow the Production CA private key be compromised. This document
shows the steps used to create both the Root CA and the Production CA, as well
as to perform common options such as creation and revocation of certificates
with the Production CA. The operations have been encapsulated in the ca
shell script.
There is also a Test CA, used for generating certificates for servers in the dev, staging, and sandbox environments, but this CA is completely independent of the Production CA and its certificates will not be accepted in the production environments. The Test CA has the same hierarchy as the Production CA, with a Root CA delegating certificate signing and management to an intermediate CA.
New certificates created using the CA have two components: the certificate and the private key:
publish_cert_orcid in the
"Use" section, below), any local copies of the private key should be
deleted. (We no longer keep copies of these Node keys, since a
new cert and key can be generated easily if a key is lost or compromised.)The private keys needed to issue certs are contained in a binary sparsebundle file. Contact DataONE root system administrators for access.
VERY IMPORTANT! Since merge commits are not possible with a binary file, ALWAYS...
1) Pull the latest version of the sparsebundle before starting any changes. 2) Inform other certificate admins on slack that you are working in the bundle. 3) Push your sparsebundle changes immediately, and inform the other admins when you're done.
The CA scripts rely on the BASH shell and are developed on OS X (bash 3.2.53) though should work without modification on Linux. Dependencies are:
sed, awk, cut, sort, gitInstalling the DataONE CA involves the following steps. In these instructions,
it is assumed that the CA software is being installed in
${HOME}/Projects/DataONE/tools, identified by ${CA_HOME} in the examples.
Adjust as appropriate for your system.
The CA is distributed from GitHub. Checkout the tool to the desired location
cd ${HOME}/Projects/DataONE/tools
git clone git@github.com:DataONEorg/ca.git
export CA_HOME="$(pwd)/ca"Create a symbolic link for /var/ca to the checkout location:
sudo ln -s ${CA_HOME}/ca /var/caMount the encrypted volume using Finder or the command line when ready to create certificates or update the revocation list:
hdiutil attach -agentpass /path/to/encrypted/DMGNote -- Do not keep the encrypted volume with the keys on your laptop or any other device that is regularly connected to the Internet. Keep it on a USB stick or some other physical media that can be disconnected.
Verify the installation by running the cert_status script:
cd ${CA_HOME}
./cert_statusIf all is good, then the script will examine the contents of the Test Environment certificates folder and report on the status of each .pem file found there. The output is lengthy, and looks something like:
{"what":"Certificate Status",
"Generated":"2015-03-18T11:23:26.000+00:00",
"content":[
{
"File_name" : "/Users/vieglais/Projects/DataONE_61385/svn/software/tools/trunk/ca/DataONETestIntCA/certs/cn-dev-orc-1.pem",
"Author" : "",
"Serial" : "DA3263A2A12D004B",
"DN" : "DC=org,DC=dataone,DC=test,CN=cn-dev-orc-1",
"Created" : "2012-07-24T03:39:45.000+00:00",
"Expires" : "2015-07-24T03:39:45.000+00:00",
"Expires_days": 127,
"Revocation" : "Revoked",
"Validity" : "Valid"
}
,
{
"File_name" : "/Users/vieglais/Projects/DataONE_61385/svn/software/tools/trunk/ca/DataONETestIntCA/certs/cn-dev-orc-1.test.dataone.org-1.pem",
"Author" : "",
"Serial" : "DA3263A2A12D0055",
"DN" : "DC=org,DC=dataone,CN=cn-dev-orc-1.test.dataone.org",
"Created" : "2012-07-27T21:25:34.000+00:00",
"Expires" : "2015-07-27T21:25:34.000+00:00",
"Expires_days": 131,
"Revocation" : "Revoked",
"Validity" : "Valid"
}
...and so on.
Four shell scripts are included to assist with certificate management:
ca: This is the main script for creating and revoking certificates.
cert_status: This script reports the status for a single certificate or all certificates
in an environment.
publish_cert and publish_cert_orcid: Provide a convenient mechanism for packaging a certificate
and key, and placing them in a secure location for download by an authenticated user.
The publish_crl script (for publishing the certificate revocation list to the CRL servers) has
been move to the SHA-1_ARCHIVE directory. In practice no clients rely on the CRL -- see
this blog post for more explanation. If you still
need details of how to use the publish_crl script, see the
original README file
caThe shell program ca can be used to manage certificates from both the Test
CA and the Production CA. It determines which CA to use based on commandline
arguments. Type ./ca -h to see the usage help for the ca utility.
To install the DataONE certificate authority, simply:
1) install openssl on your machine
2) Check out a working copy of the CA from the DataONE GitHub repository
3) Mount the private key encrypted volume under /Volumes/DataONE
The ca utility can create, revoke, and display certificates, and can
generate the Certificate Revocation List (CRL) for either of the CAs. Examples
follow:
To create a Production certificate for the MN with nodeid "KNB":
./ca -c Prod urn:node:KNBTo display a Production certificate for the MN with nodeid "KNB":
./ca -d Prod urn:node:KNBTo revoke a Production certificate for the MN with nodeid "KNB":
./ca -r Prod urn:node:KNBTo generate a CRL for the Prod CA:
./ca -g ProdAny of these commands can be made to work on the Test CA instead by switching
Prod to Test.
Once new CSRs, Certificates, and CRLs have been generated, they should be added to GitHub and all modified files should be checked in to GitHub so that others managing the CA can access all the updated content. The only exception are the private keys that are generated, which should be given to the MN operator along with instructions on how to protect the private key. The private key should be deleted from the CA to avoid possible exposure of the keys.
cert_statusThe script cert_status provides a mechanism to report on the status of a
single certificate or all certificates within the Production or Test
environments. Report output is in JSON or pipe (|) separated values and
includes the attributes:
File_name: Full path to the certificateAuthor: The name of the GitHub user that checked in the certificateSerial: The certificate serial numberDN: The certificate Distinguished NameCreated: Indicates when the certificate was createdExpires: Indicates when the certificate will expireExpires_days: Number of days until the expiration dateRevocation: Indicates if the certificate appears in the revocation listValidity: Indicates if the test openssl verify passes.cert_status can also be used to generate VCalendar .ics files, one for
Production and one for the Test environment, that includes dates for
certificate and revocation list expiry. These are checked in to GitHub
and can be subscribed to using Google Calendar or iCal using the calendar
locations of:
https://raw.githubusercontent.com/DataONEorg/ca/main/Prod_events.ics
for the Production environment, and:
https://raw.githubusercontent.com/DataONEorg/ca/main/Test_events.ics
for the Test environment.
Example Show status of a single certificate in test environment:
./ca cert_status -A DataONETestIntCA/certs/urn\:node\:mnTestGulfWatch.pemExample Show status of a single certificate in production environment, using the default locations for certificates and CRL:
./cert_status -A -P DataONEProdIntCA/certs/urn\:node\:GULFWATCH.pemExample Show status of a single certificate in production environment, explicitly indicating which certificates and CRL to use:
./cert_status -A \
-a DataONEProdIntCA/certs/DataONEProdIntCA.pem \
-c DataONEProdRootCA/certs/DataONEProdRootCA.pem \
DataONEProdIntCA/certs/urn\:node\:GULFWATCH.pemExample Generate a pipe delimited text file reporting on all the test certificates:
./cert_status -S > testcerts.csv; \
for f in $(find DataONETestIntCA/certs -name *.pem); \
do ./cert_status -A -s $f >> testcerts.csv; doneor:
./cert_status -s -A DataONETestIntCA/certsExample Generate a pipe delimited text file reporting on all the production certificates:
./cert_status -H > testcerts.csv; \
for f in $(find DataONETestIntCA/certs -name *.pem); \
do ./cert_status -A -s \
-a DataONEProdIntCA/certs/DataONEProdIntCA.pem \
-c DataONEProdRootCA/certs/DataONEProdRootCA.pem \
$f >> prodcerts.csv; doneor:
./ca cert_status -s -A -P DataONEProdIntCA/certsExample Generate a calendar of events in .ics format for production environment certificate expirations and the next update time for the CRL. Output is to the file "Prod_events.ics" for the production environment or "Test_events.ics" for the test environment. The calendar can be subscribed to using the respective GitHub URL:
./ca cert_status -P -Lpublish_cert and publish_cert_orcidThe scripts publish_cert and publish_cert_orcid each provide a convenient mechanism to
package a certificate, its key, and the CSR used to generate the certificate into a .zip
file and upload it to the distribution server (currently https://project.dataone.org/).
The script accepts two arguments:
publish_cert, orpublish_cert_orcid (but only the numerical part; see below).certs
folder of the respective CA.Note -- The resulting file names have the ":" character replaced with "_".
The script uses ssh to connect to the distribution host, create a target
folder if necessary, and upload the package .zip file. As such, it is
necessary for the user running the script to have SSH access to the
distribution host and write access to the destination folder (/var/www/users).
Example Share a certificate and key for user vieglais:
# For LDAP:
./ca publish_cert vieglais DataONETestIntCA/certs/urn:node:ATestCert.pem
# or for ORCID:
./ca publish_cert 0000-0002-6513-4996 DataONETestIntCA/certs/urn:node:ATestCert.pemNote -- Only the numerical part of the ORCID should be used, as shown in the example above!
The resulting package would be downloadable from:
https://project.dataone.org/~vieglais/urn_node_ATestCert.zip
After unzipping, the result would be:
urn_node_ATestCert/
info.txt
urn_node_ATestCert.pem
urn_node_ATestCert.csr
private/
urn_node_ATestCert.keyThe file info.txt contains general information about the certificate
generated by the cert_status program.
OpenSSL was used to create the various CA files and operate the CA. The
following sections are a synopsis of how all the CAs were created and how
various CA functions can be run using OpenSSL alone. The ca shell script
automates some of these functions (most notably for Node certificate creation),
so their inclusion here is mainly as a reference and not intended for typical usage.
For more information on OpenSSL, see openssl.org. For more detail
on the configuration files (openssl.cnf or openssl.tmpl), see
the openssl documentation
or this Openssl.conf walkthrough
The original CA certificates were generated using SHA-1, which is now considered insecure. In 2024, therefore, new SHA-256-encrypted Root CAs were generated, and these were used to "cross-sign" the intermediate certs. This process entailed generating new Intermediate CA certs for Production and Test, while ensuring:
See Appendix 2 for a brief overview of how Cross Signing works
At the same time, all the old SHA-1 contents of this repo were moved into the SHA-1_ARCHIVE
subdirectory (see Appendix 3). A new, clearer and more-consistent
naming convention was then adopted for the new directories and files, as follows:
PRODUCTION TEST
-----------------------------------------------------------------------------------
### ROOT ###
directory name: DataONEProdRootCA DataONETestRootCA
cert name: DataONEProdRootCA.pem DataONETestRootCA.pem
CN = "DataONE Prod Root CA" "DataONE Test Root CA"
### INTERMEDIATE ###
directory name: DataONEProdIntCA DataONETestIntCA
cert name: DataONEProdIntCA.pem DataONETestIntCA.pem
(see Note) CN = "DataONE Production CA" "DataONE Test Intermediate CA"NOTE: The Intermediate CNs are inconsistent because the Subjects (and therefore the CN values) for the Intermediate certs must match those in the old SHA-1 root CAs, in order for cross-signing to work.
If we have the opportunity to change the Intermediate CNs in the future, we can make them consistent
by renaming "DataONE Production CA" to "DataONE Prod Intermediate CA".
# Production
DC=org, DC=dataone, CN=DataONE Prod Root CA
DC=org, DC=dataone, CN=DataONE Production CA
#Test
DC=org, DC=dataone, CN=DataONE Test Root CA
DC=org, DC=dataone, CN=DataONE Test Intermediate CA DC=org, DC=dataone, CN=urn:node:SOMENODE 100 years 3 years mkdir /var/ca
cd /var/ca
mkdir DataONEProdRootCA
cd DataONEProdRootCA
mkdir certs newcerts private req
touch index.txt
# Edit the openssl.cnf config file if needed; e.g. check the 'dir' entry in [ CA_default ].
openssl req -new -newkey rsa:4096 -keyout /Volumes/DataONE/DataONEProdRootCA.key \
-out req/DataONEProdRootCA.csr -config ./openssl.cnf
# You will be prompted for:
# 1. a passphrase to set for the new key, and
# 2. the Common Name (CN) to set
openssl ca -create_serial -out certs/DataONEProdRootCA.pem -days 36500 \
-keyfile /Volumes/DataONE/DataONEProdRootCA.key -selfsign -config ./openssl.cnf \
-extensions v3_ca -infiles req/DataONEProdRootCA.csr
# You will be prompted for the passphrase for the existing (prod root) key cd ..
mkdir DataONEProdIntCA
cd DataONEProdIntCA
mkdir certs newcerts private req
touch index.txt
# No need to edit the config file; uses the one from the root CA
###
# This is how we did it originally:
# ### OMIT FOR CROSS SIGNING ###
# openssl req -new -newkey rsa:4096 -keyout /Volumes/DataONE/DataONEProdCA.key \
# -out req/DataONEProdIntCA.csr -config ../DataONEProdRootCA/openssl.cnf
# # You will be prompted for:
# # 1. the Common Name (CN) to set, and
# # 2. a passphrase to set for the new key
# ### END OMIT FOR CROSS SIGNING ###
#
# However, for cross-signing, we should NOT generate a new key (" -newkey "),
# but instead re-use the original intermediate key...
###
openssl req -new -key /Volumes/DataONE/SHA-1_ARCHIVE/DataONEProdCA.key \
-out req/DataONEProdIntCA.csr -config ../DataONEProdRootCA/openssl.cnf
# You will be prompted for:
# 1. the passphrase for the existing (prod intermediate) key, and
# 2. the Common Name (CN) to set (NOTE for cross-signing, this MUST match the CN used in the old
# intermediate cert!)
cd ../DataONEProdRootCA
openssl ca -out ../DataONEProdIntCA/certs/DataONEProdIntCA.pem -days 36500 \
-keyfile /Volumes/DataONE/DataONEProdRootCA.key -config ./openssl.cnf \
-extensions v3_ca -infiles ../DataONEProdIntCA/req/DataONEProdIntCA.csr
# You will be prompted for the passphrase for the existing (prod root) key cd ..
cat DataONEProdRootCA/certs/DataONEProdRootCA.pem \
DataONEProdIntCA/certs/DataONEProdIntCA.pem > DataONECAChain.crt
# ...and similarly for Test certsNOTE - in addition to the DataONE Root and Intermediate CA certs, the Prod and Test CA chain files currently (Jan 2024) also include the following 3 CILogon CA certs, for legacy reasons. (these were copied across from the old SHA-1 cert chains):
subject=DC = org, DC = cilogon, C = US, O = CILogon, CN = CILogon Basic CA 1
issuer=DC = org, DC = cilogon, C = US, O = CILogon, CN = CILogon Basic CA 1
subject=DC = org, DC = cilogon, C = US, O = CILogon, CN = CILogon OpenID CA 1
issuer=DC = org, DC = cilogon, C = US, O = CILogon, CN = CILogon OpenID CA 1
subject=DC = org, DC = cilogon, C = US, O = CILogon, CN = CILogon Silver CA 1
issuer=DC = org, DC = cilogon, C = US, O = CILogon, CN = CILogon Silver CA 1 cd DataONEProdIntCA
openssl genrsa -passout pass:temp -des3 -out private/NodeNPass.key 2048
openssl rsa -passin pass:temp -in private/NodeNPass.key -out private/NodeN.key
rm private/NodeNPass.key
# NOTE: It's best to use the ca script to do this, because there isn't
# an openssl.cnf file in this directory - only a template
openssl req -config ./openssl.cnf -new -key private/NodeNPass.key -out req/NodeN.csr
# You will be prompted for the Common Name (CN) to set
openssl ca -config ./openssl.cnf -create_serial -days 1095 \
-out certs/NodeN.pem -infiles req/NodeN.csr
# You will be prompted for the key passphraseIf a certificate signing request is provided, then it can be signed as follows:
cd DataONETestIntCA
openssl ca \
-config openssl.csr_ca.conf
-subj "/DC=org/DC=dataone/CN=NODEID" \
-preserveDN -batch \
-notext \
-create_serial \
-days 1095 \
-out csr/NODEID.pem \
-infiles csr/NODEID.csr.pemWhere NODEID is the node identifier.
openssl ca -config ./openssl.cnf -revoke certs/NodeN.pem mkdir /var/ca
cd /var/ca
mkdir DataONETestRootCA
cd DataONETestRootCA
mkdir certs newcerts private req
touch index.txt
# Edit the openssl.cnf config file if needed; e.g. check the 'dir' entry in [ CA_default ].
openssl req -new -newkey rsa:4096 -keyout /Volumes/DATAONE/DataONETestRootCA.key \
-out req/DataONETestRootCA.csr -config ./openssl.cnf
# You will be prompted for:
# 1. a passphrase to set for the new key, and
# 2. the Common Name (CN) to set
openssl ca -create_serial -out certs/DataONETestRootCA.pem -days 36500 \
-keyfile /Volumes/DATAONE/DataONETestRootCA.key -selfsign -config ./openssl.cnf \
-extensions v3_ca -infiles req/DataONETestRootCA.csr
# You will be asked for the key passphraseThis is a cross-signed intermediate cert, in that it has the same subjectDN and public key as the original DataONETestIntCA, but it is signed by the new sha256-based DataONETestRootCA.
cd /var/ca
mkdir DataONETestIntCA
cd DataONETestIntCA
mkdir certs newcerts private req
touch index.txt
# No need to edit the config file; uses the one from the root CA
###
# This is how we did it originally:
# ### OMIT FOR CROSS SIGNING ###
# openssl req -new -newkey rsa:4096 -keyout /opt/DataONE/DataONETestIntCA.key \
# -out req/DataONETestIntCA.csr -config ../DataONETestCA/openssl.cnf
#
# # You will be prompted for:
# # 1. the Common Name (CN) to set, and
# # 2. a passphrase to set for the new key
# ### END OMIT FOR CROSS SIGNING ###
#
# However, for cross-signing, we should NOT generate a new key (" -newkey "),
# but instead re-use the original intermediate key...
###
openssl req -new -key /Volumes/DATAONE/SHA-1_ARCHIVE/DataONETestIntCA.key \
-out req/DataONETestIntCA.csr -config ../DataONETestRootCA/openssl.cnf
# You will be prompted for:
# 1. the passphrase for the existing (test intermediate) key, and
# 2. the Common Name (CN) to set (NOTE for cross-signing, this MUST match the CN used in the old
# intermediate cert!)
cd ../DataONETestRootCA
openssl ca -out ../DataONETestIntCA/certs/DataONETestIntCA.pem -days 36500 \
-keyfile /Volumes/DATAONE/DataONETestRootCA.key -config ./openssl.cnf \
-extensions v3_ca -verbose -infiles ../DataONETestIntCA/req/DataONETestIntCA.csr
# You will be prompted for the passphrase for the existing (test root) key
# Create DataONETestIntCA/serial with serial number of the DataONETestIntCA.pem + something cd /var/ca
cat DataONETestCA/certs/DataONETestCA.pem \
DataONETestIntCA/certs/DataONETestIntCA.pem > DataONETestCAChain.crtWhen we started issuing DataONE node certificates in 2012, we were using SHA-1-encrypted Root and Intermediate CA certs. Since then, SHA-1 has widely been recognized as insecure, and has been replaced with SHA-256. However, since it would be a huge task to re-issue all the node certificates currently in use, we need a way of upgrading our CA certs to SHA-256, whilst keeping them backwards-compatible with existing node certs. This can be done by a process known as Cross Signing. (For an excellent overview of how cross signing works, see Scott Helme's blog).
Basically, here's what happens when a DataONE Node cert (or any cert, for that matter) is created:
(It's interesting to note that this process does not require a root CA cert or an Intermediate CA cert; only the Intermediate's private key is needed).
Later, when a DataONE Node cert is validated against the DataONE Intermediate CA cert (from the cert chain on the server), 2 things are checked:
the signature on the bottom of the Node cert is decrypted using the Intermediate CA's public key. This tells us that if the CA's public key can decrypt it, the CA's private key must have encrypted it, so authenticity has been verified.
the server then calculates its own hash of the Pre-Certificate to compare to the hash stored in the signature and determine if they are identical. If they match, the certificate has not been tampered with, so Integrity has been verified.
Now we know we can trust the contents of the Node cert, authentication can be completed by simply verifying that the "Issuer" field in the Node cert matches the "Subject" field in the Intermediate cert.
So - in summary - only 2 pieces of information from the Intermediate certificate are used to authenticate the Node cert:
Therefore, it is possible to have two (or multiple) different versions of the Intermediate cert, provided they each contain the same public key and the same Subject!
This is why cross signing is possible.
So - all we need to do, in order to create a cross-signed SHA-256 Intermediate is:
Now we can replace the old sha-1 CA cert chain with this new SHA-256-based CA chain, and it will work with all the existing Node certificates and any new Node certs issued against the new CA.
(Final note; "Cross signing" is a misnomer. it implies that one intermediate certificate is signed by two different Root CAs. This is not the case, as can be seen above.)
── ca
├── DataONEProdIntCA
├── DataONEProdRootCA
├── DataONETestIntCA
├── DataONETestRootCA
└── SHA-1_ARCHIVE ## SHA-1 content from old top-level ca directory moved here:
├── DataONEProdCA ## old SHA-1 Prod Intermediate CA
├── DataONERootCA ## old SHA-1 Prod Root CA
├── DataONETestCA ## old SHA-1 Test Root CA
└── DataONETestIntCA ## old SHA-1 Test Intermediate CA