ergonomics

[dgreisen]

Couple things:

• I initially proposed a .stele folder in root of library. However, I'm now thinking it should be .stelae, since this is the root of a folder with many stele in it. This would leave open the posibility of including a .stele file in individual stele.
• I would like the keystore to be at .stelae/keystore, that way you never have to specify keystore, it will just use whatever keys are in that keystore. if you want to specify a different keystore, you could.
• ideally, all commands in taf and stele would use the library that the command was called from.
• there is already a cargo container called stele. should we change our name to stelae?

Proposed Stelae command structure:

• stelae
  • list
  • should work on an archive, stele, target
  • data: up to date with remote, uncommitted targets, unsigned targets, which targets are cloned/not cloned locally, bare/normal; display in pretty tree with optional json output, list dependencies
  • stele
  • add - add a existing stele as a new dependent stele and sign
  • remove - remove an existing dependency and sign
  • list - appropriate subset of root list command
  • init - initialize a new stele and add to dependencies
  • target
  • add - add a new target repository and sign
  • remove - remove an existing target repository and sign
  • list - appropriate subset of root list command
  • sign - sign a stele, or all stele in an archive; error if uncommitted changes or not up-to-date with remote
  • serve - serve via http (run publish server)
  • shallow-clone - create a new archive, shallow clone root and all dependency stelae, can include/exclude target types to clone.
  • clone - create a new archive, clone root stelae and all dependencies.
  • update - update a particular archive or stele, update all dependencies
  • validate - validate a particular archive or stele, validate all dependencies
  • init - initialize a new archive with a new root stele
  • upgrade - upgrade to a new version of stelae
  • key management...

[dgreisen]

I'm moving over discussion from another issue:

from @n-dusan:

Right now we use overloaded terms such as repository (in taf), library (in taf, platform, stele), dependency (in taf), host (in taf), namespace (publish-client, taf), environment (publish-server) in different modules that mean different things in platform. It gets really confusing really quickly to talk about these names without specifying which part of the stack they are referring to. We should enumerate (and update) this list of confusing names with a proposed renames and discuss. After, it also might make sense to update platform documentation with naming conventions that we adopted explaining the reasoning behind each of them.

Current list of overloaded terms and what they refer to:

• repository
  • git repository
  • taf repository that can be:
    • authentication (law) repository
    • other, linked (target) repositories
  • ...
• library
  • list of taf/git/stele repositories
  • ...
• dependencies
  • list of taf/stele repositories which can host other authentication (law) repositories specified in dependencies.json
  • but can also refer to:
  • python dependencies
  • frontend npm dependencies
  • ...
• host
  • terraform/nginx host
  • a library that hosts other libraries (in taf specified by hosts.json)
  • ...
• namespace
  • publish client namespace relates to partner id and is used for example here
  • taf partner namespace which consists of org_name/repo_name (see mirrors.json)
  • ...
• environment
  • in publish server, refers to production, development and preview types of partner repositories (https://github.com/cityofsanmateo/law-html is considered production while https://github.com/cityofsanmateo/law-html-preview is considered preview environment.
  • can also mean dev and production` AWS servers (environments)
  • ...

Discuss a naming nomenclature which would rename some of these terms. Then incrementally refactor existing code to new naming conventions

[dgreisen]

from @renatav:

Just some small clarifications and nuances:

• repository in my mind is always a git repository and I use this term when it is not important if it is an authentication repository or a target. If the distinction is important, I'd say auth/law repo or target/html/codified etc. repo
• library in my mind is a top level folder/root, but we it's actually a collection of the top level repositories and their target repositories
• host in my mind is always nginx host
• namespace - this is a tricky one, a partner's namespace is always just one word, like cityofsanmateo. A repository's name is namespace/some_name. This is used when building reader wheels. It's not the same as partner's id. This is not great

[dgreisen]

from @dgreisen

My suggestions:

• archive - library is overloaded, so we could change the name of the folder containing all the repositories an archive
• stele - we don't have a name for a collection of auth repo and target repos. I would vote this be called a stele.
• main stele - the stele that contains info about all the other stele in the archive (would love a better name)
• referenced stele - the stele that are incorporated by reference into another stele through its stelae file (would love a better name)
• org - I say we just adopt the github nomenclature of <org>/<repo name>
• repository - always a git repository that can be a:
  • auth/law repo
  • target repo

[dgreisen]

target repo is really not a good name. it is meaningless to almost everybody except people familiar with tuf. No idea what a better name is, though.

[n-dusan]

We've changed the repository name to stelae now.

target repo is really not a good name. it is meaningless to almost everybody except people familiar with tuf. No idea what a better name is, though.

Agree, we could maybe use git's submodule nomenclature? https://git-scm.com/book/en/v2/Git-Tools-Submodules Or maybe even module. So target repo would be a stele (sub)module.

[n-dusan]

Meeting notes from 2/17

w/ @dgreisen @renatav @tombh @n-dusan

Naming

We've reached a naming consensus. We will be using:

• archive instead of library
• stele for collection of authentication and target repositories
• root stele for stele that contains info about all the other stele in the archive
• child stele for stele that are incorporated by reference into another stele through its stelae file
• <org>/<repo name>
• data repository instead of target repository. target was a naming convention we took from tuf and is unrelated to Stelae users.
• environment, which must be either preview, development or production.

Stelae command structure

Most of Stelae commands listed here should be ported from TAF. At the time being, we should utilize taf commands for mutating and make Stelae commands as wrappers around taf. Reasoning behind this is that taf uses python-tuf for signing repositories. While tuf community does have a Rust equivalent of python-tuf, it's still in early stages of development.

For reading the state of stele repositories, we should implement native Rust commands.

Commands:

• Written in Rust natively:

  • stelae
  • list
    • should work on an archive, stele, data (target)
    • data: up to date with remote, uncommitted targets, unsigned targets, which targets are cloned/not cloned locally, bare/normal; display in pretty tree with optional json output, list dependencies
  • stele
    • list - appropriate subset of root list command
  • data (what we used to call target)
    • list - appropriate subset of root list command
  • serve - serve via http (run publish server)
  • upgrade - upgrade to a new version of stelae

• Written in Rust as wrappers around taf commands:

  • stelae
  • stele
    • add - add a existing stele as a new dependent stele and sign
    • remove - remove an existing dependency and sign
    • init - initialize a new stele and add to dependencies
    • data (what we used to call target)
    • add - add a new target repository and sign
    • remove - remove an existing target repository and sign
  • sign - sign a stele, or all stele in an archive; error if uncommitted changes or not up-to-date with remote
  • shallow-clone - create a new archive, shallow clone root and all dependency stelae, can include/exclude target types to clone.
  • clone - create a new archive, clone root stelae and all dependencies.
  • update - update a particular archive or stele, update all dependencies
  • validate - validate a particular archive or stele, validate all dependencies
  • init - initialize a new archive with a new root stele
  • key management...

[n-dusan]

Proposed workflow for an end user who wants to easily (1) authenticate a repo, keep it up-to-date

• wget https://github.com/.../releases/latest
• ./stelae clone https://github.com/DCCouncil/law --init-archive=c:\dgreisen\archive --trust-server
  • initialize archive (make final directory (only) if it doesn't exist, nice error otherwise)
  • clone dccouncil/law
  • update database
  • make it default stele
• cd archive
• ./stelae serve
• ./stelae update
  • taf repo update default stele
  • update db
• ./stelae rebuild-db
• ./stelae upgrade