Trialing asdf for repo specific tool management

[scottmuc]

I'm starting to grow some machine specific configuration management setup that is creating a spread of Not Invented Here (NIH) installers. Rather than having a language version manager for every language, asdf can be the one to rule them all. My bias is usually for language specific things, but I feel this might be a good level of abstraction.

Things I want to see if asdf can manage for me are:

• nodejs
• golang
• terraform
• python
• ruby

I much prefer project level scope tool declaration over machine level.

asdf project page: https://asdf-vm.com/

[scottmuc]

Pleasantly surprised by the plugin collection: https://github.com/asdf-vm/asdf-plugins

The plugins look easy to understand and have well defined interfaces. Here's a plugin that can install multiple different tools, but doesn't feel too complex in doing so: https://github.com/vmware-tanzu/tanzu-plug-in-for-asdf

[scottmuc]

Testing with terraform. I uninstalled the version that was installed via apt:

~/workspace/infrastructure/homedirs/wsl ? terraform -v
zsh: command not found: terraform
~/workspace/infrastructure/homedirs/wsl ? asdf install terraform latest
Downloading terraform version 1.3.8 from https://releases.hashicorp.com/terraform/1.3.8/terraform_1.3.8_linux_amd64.
zip
Verifying signatures and checksums
gpg: keybox '/tmp/asdf_terraform_dbFJ4Z/pubring.kbx' created
gpg: /tmp/asdf_terraform_dbFJ4Z/trustdb.gpg: trustdb created
gpg: key 34365D9472D7468F: public key "HashiCorp Security (hashicorp.com/security) <security@hashicorp.com>" importe
d
gpg: Total number processed: 1
gpg:               imported: 1
gpg: Signature made Thu Feb  9 20:43:58 2023 CET
gpg:                using RSA key 374EC75B485913604A831CC7C820C6D5CD27AB87
gpg: Good signature from "HashiCorp Security (hashicorp.com/security) <security@hashicorp.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: C874 011F 0AB4 0511 0D02  1055 3436 5D94 72D7 468F
     Subkey fingerprint: 374E C75B 4859 1360 4A83  1CC7 C820 C6D5 CD27 AB87
terraform_1.3.8_linux_amd64.zip: OK
Cleaning terraform previous binaries
Creating terraform bin directory
Extracting terraform archive
~/workspace/infrastructure ? asdf list
terraform
  1.3.8
~/workspace/infrastructure ? terraform -version
No version is set for command terraform
Consider adding one of the following versions in your config file at
terraform 1.3.8
~/workspace/infrastructure ? asdf local terraform latest
~/workspace/infrastructure ? terraform -version
Terraform v1.3.8
on linux_amd64
~/workspace/infrastructure ? cat .tool-versions
terraform 1.3.8

I love that the shim knows that a version of terraform has been installed, but won't wrap it unless explicitly told to. I really like this.

This does move more bootstrapping to the repository level which is an adequate tradeoff for me. A repaved machine with asdf could mean that I have a machine that's capable of bootstrapping all repositories I'm concerned with and are asdf enabled (having a .tool-versions in the root of the repository).

[scottmuc]

So far some good with terraform. Gives me something to dig into. The disk layout is easy to follow as well:

~/.asdf ? ls
CHANGELOG.md     README.md    asdf.fish  ballad-of-asdf.md  defaults   help.txt  plugins  test
CONTRIBUTING.md  SECURITY.md  asdf.nu    bin                docs       installs  scripts  version.txt
LICENSE          asdf.elv     asdf.sh    completions        downloads  lib       shims
~/.asdf ? tree plugins
plugins
└── terraform
    ├── LICENSE
    ├── Makefile
    ├── README.md
    ├── bin
    │   ├── install
    │   ├── list-all
    │   ├── list-legacy-filenames
    │   └── parse-legacy-file
    ├── hashicorp.asc
    └── test
        ├── install.bats
        └── parse-legacy-file.bats

3 directories, 10 files
~/.asdf ? tree installs
installs
└── terraform
    └── 1.3.8
        └── bin
            └── terraform

3 directories, 1 file
~/.asdf ? tree shims
shims
└── terraform

0 directories, 1 file
~/.asdf ? cat shims/terraform
#!/usr/bin/env bash
# asdf-plugin: terraform 1.3.8
exec /home/wsl/.asdf/bin/asdf exec "terraform" "$@" # asdf_allow: ' asdf '
~/.asdf ? cd
~ ? which terraform
/home/wsl/.asdf/shims/terraform
~ ? terraform -v
No version is set for command terraform
Consider adding one of the following versions in your config file at
terraform 1.3.8
~ ? j infra
/home/wsl/workspace/infrastructure
~/workspace/infrastructure ? terraform -v
Terraform v1.3.8
on linux_amd64

I tend to swing between the shim approach vs an update-path approach. Both are offering a level of indirection. Reading .tool-versions for every execution makes the shim contract explicit.

[scottmuc]

Given that .tool-versions will pin versions. I'll need to add update tools to the repaving process to ensure that I don't get stuck with pinned old versions for too long and never upgrade.

[scottmuc]

Tagging @willoleary6 in order to send my spammy updates :-D

[scottmuc]

Python setup worked, but was confusing because of my existing install. Installing python via asdf required some packages to be installed such as libssl-dev libreadline-dev libffi-dev

A couple wsl repaves ought to flesh out what needs to be updated in the setup automation. Before writing an asdf bootstrap script, I'll manually document the steps in the repave template for now.

For now, I have python and a working ansible setup.

Something to look into is if a machine wide python3 installation is still necessary.

[scottmuc]

OK with python I need to have a $HOME/.tool-versions python and one for projects. Things get confusing when there's a python dependency for the project, and there's a python dependency for my general unix tool-chain. The aim here is to minimize the system level python which gets installed as a dependency for autojump. This is starting to make things lot harder to understand compared to having a global system python. Something I'll continue to play with, but python might be a harder one to isolate because it's needed in different contexts.

[scottmuc]

Versioning nodejs with asdf proved successful with https://github.com/devopsbookmarks/devopsbookmarks.com/commit/069dbc5ccbd49c123ea977420b8229ab4afa2239

I do wonder if this will have similar issues as python where there are general purpose shell tools that depend on nodejs and might break if I am switching between projects.

[scottmuc]

Something I'm finding with asdf is that it removes a lot of cognitive load of learning/understanding multiple tool version managers. I can invest in learning asdf well, and that knowledge will assist with all tool versioning problems I might face. There will be some idiosyncratic issues, but those will hopefully be the exception, not the norm.

[scottmuc]

That above commit that removes a custom installer is a good demonstration of how concise asdf configuration can become. I'm becoming a fan.

[scottmuc]

Summary

After a little over a day tinkering with asdf, I'm happy to continue to use it in the foreseeable future. It helped me delete a lot of code with an added dependency that I feel has a strong community around it. I really like the lower cognitive load of trying to understand 1 tool version manager over tool specific ones.

It's helping me view my workstation automation at 3 levels:

1. System level - these are managed via the OS package management and will be global to the host. Useful for things that are hard to scope to a project. Things like having zsh installed fit here.
2. User level - things to be use across all projects, or not even project related. This is where asdf gets installed. Same with things like 1 Password. My computer wide tool-belt sits here. asdf could possibly manage these to with $HOME/.tool-versions.
3. Project/Repo level - language runtimes and project specific. This repo has a good example where terraform, ansible and gum are only accessible for doing things here. Running those commands outside of this repo's directory structure results in a helpful message from asdf saying that that shim has no version of the tool to use, so it does nothing.

The result is that my machine configuration doesn't reveal too much about what projects I'm working on (not that I'm trying to hide it, it's more about configuration living in the appropriate level).

My WSL config reflects this well now I think:

main() {
  configure_wsl
  configure_dns
  symlink_all_the_things
  install_os_packages
  default_to_zsh
  setup_neovim
  setup_ssh_dir
  install_asdf
  install_all_things_docker
  install_git_duet "0.9.0"
  install_delta "0.14.0"
  install_1password_cli "2.12.0"
  install_alacritty_colorscheme "1.0.1"
}

It's mainly system and user level stuff being managed there giving me a platform to work on the relevant project repositories.

For my other custom code, I can see turning them into asdf plugins might help the broader community of reuse.