From testdata to pypeline tests

[vanrein]

Current setups are made with testdata (also in the Docker Demo) and tests may be run against a live TLS Pool setup like that.

With pypeline in place, we might run the entire TLS Pool, with prior setup for databases and SoftHSM, from scratch for any single test. This takes a fair bit of work/changes.

Pypelining the Entire Thing:

• SOFTHSM_CONF points to a conffile for libsofthsm2.so

• Generate this file with FILE:WHATEVER filenames from Pypeline

• Setup BDB with a fresh DB-context and run have_db

• Introduce keys and certificates by recipe (template, commandlist)

• Generate a corresponding tlspool.conf (template expansion?)

New Requirements for Pypeline:

• Pypeline: Setup for envvars, beginning of line?
  ENV:NAME VALUE or simply NAME=VALUE

• Added tool or builtin feature: Expansion of templates to configfiles:
  tmpl2file test13.tmpl FILE:TLSPOOL_CONF varnm value... (when it is builtin, it could also be used for the cmdline args)

Key gen in testdata:

• PRIVKEY1=$(shell $(P11TOOL) --list-privkeys '$(P11URI)' | sed -e '/object=obj1label/!d' -e 's/^[ \t]*URL: //')

• $(P11TOOL) --generate-rsa --bits $(PGPRSABITS) --label=obj1label --id=3031 --outfile=/dev/null '$(P11URI)'

• Choose:

  • $(CERTTOOL) --outfile $@ --outder --generate-self-signed --load-privkey='$(PRIVKEY3)' --template=$<

  • $(CERTTOOL) --outfile $@ --outder --generate-certificate --load-ca-certificate=tlspool-test-ca-cert.pem --load-ca-privkey='$(PRIVKEY5)' --load-privkey='$(PRIVKEY4)' --template=$<

• $(CERTTOOL) --certificate-info --infile $@ --inder --outfile $(@:.der=.pem)

• Problem: Load the CA’s PRIVKEY from one script into another (or can we dictate its pkcs11: URI?)

  • We seem to get away without the serial number: p11tool --login --provider=/usr/lib/softhsm/libsofthsm2.so --info 'pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;token=TLS_Pool_dev_data;id=%30%35;object=obj5label;type=private'

[vanrein]

Then, use the same mechanism to add contrib scripts for use with the TLS Pool to other programs, like KXOVER or KIP. Given a basic (empty) TLS Pool these should find it easy to setup credentials, even if it may be ignorant of real-life extensions such as ACME or DANE.

[vanrein]

We should use a management file, perhaps in JSON, to administer data outside of the TLS Pool, strictly for managing the process flow. The following groups of commands would be useful.

As an implementation platform, Python is highly portable, and highly pluggable. We might use a arpa2cmd to form an interactive shell, possibly with command completion and at the very least syntax hints and decent builtin help.

Key Management

• key generate <keyid> <label>...
• key forget <keyid>...
• key cleanup
• key list (of keys)
• key export <keyid> (of public key info)
• key advise (on what to do)

Certificate Requests

• csr request <keyid> <reqid> [<contact> [<descr>]]
• csr list (of CSRs)
• csr rename <reqid> <reqid>
• csr declined <reqid> <errstr>
• csr approved <reqid> <certfile>
• csr forget <reqid>...
• csr advise (on what to do)

ACME Processing

Use an external tool. There are plenty!

The arpa2dns shell can handle the DNS portion, if so required.

DANE Processing

Use an external tool. There are not enough!

The arpa2dns shell can handle the DNS portion, including timing feedback.

Certificates

• cert install ... (TODO)