search

topgrade-rs / topgrade

Upgrade all the things
GNU General Public License v3.0
2.05k stars 139 forks source link

If unknown config values are present, configuration is silently not loaded #580

Closed daboross closed 1 year ago

daboross commented 1 year ago

Erroneous Behavior

Topgrade ignored my configuration, and proceeded to run steps I had disabled.

Using this configuration:

[misc]
# Disable specific steps - same options as the command line flag
disable = ["nix", "gcloud", "gem", "ruby_gems"]

[git]
max_concurrency = 4
# Arguments to pass Git when pulling Repositories
arguments = "--prune"

The latest topgrade outputs this:

$ cargo run -- --config ~/topgrade.toml
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `target/debug/topgrade --config /home/dross/topgrade.toml`

── 11:24:04 - System update ────────────────────────────────────────────────────
[...]

── 11:24:46 - Toolbx ───────────────────────────────────────────────────────────

── 11:24:46 - snap ─────────────────────────────────────────────────────────────
All snaps up to date.

── 11:24:48 - Firmware upgrades ────────────────────────────────────────────────
[...]

── 11:24:48 - Flatpak User Packages ────────────────────────────────────────────
Looking for updates…
Nothing to do.

── 11:24:48 - Flatpak System Packages ──────────────────────────────────────────
Looking for updates…
Nothing to do.

── 11:24:48 - Gnome Shell extensions ───────────────────────────────────────────
()

── 11:24:48 - rustup ───────────────────────────────────────────────────────────
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: checking for self-update

  stable-x86_64-unknown-linux-gnu unchanged - rustc 1.73.0 (cc66ad468 2023-10-03)

info: cleaning up downloads & tmp directories

── 11:24:48 - Cargo ────────────────────────────────────────────────────────────
[...]

── 11:24:49 - pipx ─────────────────────────────────────────────────────────────
Versions did not change after running 'pipx upgrade' for each package 😴

── 11:24:50 - Visual Studio Code extensions ────────────────────────────────────
[...]

── 11:24:53 - Kakoune ──────────────────────────────────────────────────────────
Plugins upgraded

── 11:24:53 - Containers ───────────────────────────────────────────────────────
[...]

── 11:25:10 - gcloud ───────────────────────────────────────────────────────────
Beginning update. This process may take several minutes.
ERROR: (gcloud.components.update)
You cannot perform this action because the Google Cloud CLI component manager
is disabled for this installation. You can run the following command
to achieve the same result for this installation:

sudo apt-get update && sudo apt-get --only-upgrade install google-cloud-sdk-harbourbridge google-cloud-sdk-nomos google-cloud-sdk-package-go-module google-cloud-sdk-kubectl-oidc google-cloud-sdk-config-connector google-cloud-sdk-app-engine-go google-cloud-sdk-anthos-auth google-cloud-sdk-app-engine-grpc google-cloud-sdk-log-streaming google-cloud-sdk-spanner-migration-tool google-cloud-sdk-firestore-emulator google-cloud-sdk-spanner-emulator kubectl google-cloud-sdk-minikube google-cloud-sdk-gke-gcloud-auth-plugin google-cloud-sdk-bigtable-emulator google-cloud-sdk-kpt google-cloud-sdk-enterprise-certificate-proxy google-cloud-sdk-app-engine-python-extras google-cloud-sdk-skaffold google-cloud-sdk-cbt google-cloud-sdk-pubsub-emulator google-cloud-sdk-local-extract google-cloud-sdk-cloud-run-proxy google-cloud-sdk-app-engine-python google-cloud-sdk google-cloud-sdk-cloud-build-local google-cloud-sdk-app-engine-java google-cloud-sdk-datastore-emulator google-cloud-sdk-terraform-tools

gcloud failed:
   0: Command failed: `/usr/bin/gcloud components update --quiet`
   1: `/usr/bin/gcloud` failed: exit status: 1

Location:
   src/steps/generic.rs:261
Retry? (y)es/(N)o/(s)hell/(q)uit

Expected Behavior

I expected topgrade to error out, saying my configuration was incorrect. I don't want it to be unable to load configuration, and then suddenly run steps I'd explicitly told it not to.

Even if in this case that resulted in an error, it's unsettling behavior.

Steps to reproduce

Add an unknown configuration option to your config. For example, use this configuration:

[misc]
# Disable specific steps - same options as the command line flag
disable = ["nix", "gcloud", "gem", "ruby_gems"]

[git]
max_concurrency = 4
# Arguments to pass Git when pulling Repositories
arguments = "--prune"

Make sure to disable a step that would normally run on your system.

Run topgrade. Observe that the disabled step is no longer disabled.

Possible Cause (Optional)

I expect that topgrade is ignoring all errors loading the config, rather than only ignoring errors that occur because the config file isn't present?

Problem persists without calling from topgrade

Did you run topgrade through Remote Execution

Configuration file (Optional)

[misc]
# Disable specific steps - same options as the command line flag
disable = ["nix", "gcloud", "gem", "ruby_gems"]

[git]
max_concurrency = 4
# Arguments to pass Git when pulling Repositories
arguments = "--prune"

Additional Details

Verbose Output (topgrade -v)

$ cargo run -- --config ~/topgrade.toml -v --disable cargo rustup containers kakoune pip3 pipx system snap toolbx vscode firmware gnome_shell_extensions
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `target/debug/topgrade --config /home/dross/topgrade.toml -v --disable cargo rustup containers kakoune pip3 pipx system snap toolbx vscode firmware gnome_shell_extensions`
DEBUG Version: 12.0.2
DEBUG OS: x86_64-unknown-linux-gnu
DEBUG Args { inner: ["target/debug/topgrade", "--config", "/home/dross/topgrade.toml", "-v", "--disable", "cargo", "rustup", "containers", "kakoune", "pip3", "pipx", "system", "snap", "toolbx", "vscode", "firmware", "gnome_shell_extensions"] }
DEBUG Binary path: Ok("/home/dross/src/dross/topgrade/target/debug/topgrade")
DEBUG Self Update: false
DEBUG Detected "/usr/bin/git" as "git"
DEBUG Cannot find "pwsh"
DEBUG Cannot find "powershell"
DEBUG Path "/home/dross/.config/emacs" doesn't exist
DEBUG Path "/home/dross/.emacs.d" doesn't exist
DEBUG Cannot find "doas"
DEBUG Detected "/usr/bin/sudo" as "sudo"
DEBUG Step "packer.nu"
DEBUG Step "config-update"
DEBUG Step "am"
DEBUG Step "appman"
DEBUG Step "deb-get"
DEBUG Step "pacstall"
DEBUG Step "pacdef"
DEBUG Step "protonup"
DEBUG Step "distrobox"
DEBUG Step "dkp-pacman"
DEBUG Step "Restarts"
DEBUG Step "Flatpak"
DEBUG Detected "/usr/bin/flatpak" as "flatpak"

── 11:34:17 - Flatpak User Packages ────────────────────────────────────────────
DEBUG Executing command `/usr/bin/flatpak update --user`
Looking for updates…
Nothing to do.

── 11:34:17 - Flatpak System Packages ──────────────────────────────────────────
DEBUG Executing command `/usr/bin/flatpak update --system`
Looking for updates…
Nothing to do.
DEBUG Step "Brew"
DEBUG Step "LURE"
DEBUG Step "yadm"
DEBUG Step "nix"
DEBUG Step "guix"
DEBUG Step "home-manager"
DEBUG Step "asdf"
DEBUG Step "pkgin"
DEBUG Step "bun"
DEBUG Step "zr"
DEBUG Detected "/usr/bin/zsh" as "zsh"
DEBUG Step "antibody"
DEBUG Detected "/usr/bin/zsh" as "zsh"
DEBUG Step "antidote"
DEBUG Detected "/usr/bin/zsh" as "zsh"
DEBUG Step "antigen"
DEBUG Detected "/usr/bin/zsh" as "zsh"
DEBUG Path "/home/dross/.zshrc" exists
DEBUG Step "zgenom"
DEBUG Detected "/usr/bin/zsh" as "zsh"
DEBUG Path "/home/dross/.zshrc" exists
DEBUG Step "zplug"
DEBUG Detected "/usr/bin/zsh" as "zsh"
DEBUG Path "/home/dross/.zshrc" exists
DEBUG Step "zinit"
DEBUG Detected "/usr/bin/zsh" as "zsh"
DEBUG Path "/home/dross/.zshrc" exists
DEBUG Step "zi"
DEBUG Detected "/usr/bin/zsh" as "zsh"
DEBUG Path "/home/dross/.zshrc" exists
DEBUG Step "zim"
DEBUG Detected "/usr/bin/zsh" as "zsh"
DEBUG Executing command `zsh -c '[[ -n ${ZIM_HOME} ]] && print -n ${ZIM_HOME}'`
DEBUG Command failed: Err(
   0: Command failed: `zsh -c '[[ -n ${ZIM_HOME} ]] && print -n ${ZIM_HOME}'`
   1: `zsh` failed: exit status: 1

Location:
   src/steps/zsh.rs:151)
DEBUG Step "oh-my-zsh"
DEBUG Detected "/usr/bin/zsh" as "zsh"
DEBUG Step "oh-my-bash"
DEBUG Detected "/usr/bin/bash" as "bash"
DEBUG Step "fisher"
DEBUG Step "bash-it"
DEBUG Step "oh-my-fish"
DEBUG Step "fish-plug"
DEBUG Step "fundle"
DEBUG Step "tmux"
DEBUG Step "TLDR"
DEBUG Step "pearl"
DEBUG Step "SDKMAN!"
DEBUG Detected "/usr/bin/bash" as "bash"
DEBUG Step "rcm"
DEBUG Step "maza"
DEBUG Step "apm"
DEBUG Step "fossil"
DEBUG Step "juliaup"
DEBUG Step ".NET"
DEBUG Step "choosenim"
DEBUG Step "Flutter"
DEBUG Step "go-global-update"
DEBUG Detected "/usr/local/go/bin/go" as "go"
DEBUG Executing command `/usr/local/go/bin/go env GOPATH`
DEBUG Step "gup"
DEBUG Detected "/usr/local/go/bin/go" as "go"
DEBUG Executing command `/usr/local/go/bin/go env GOPATH`
DEBUG Step "Emacs"
DEBUG Step "opam"
DEBUG Step "vcpkg"
DEBUG Step "conda"
DEBUG Step "mamba"
DEBUG Step "miktex"
DEBUG Step "pip-review"
DEBUG Step "pip-review (local)"
DEBUG Step "pipupgrade"
DEBUG Step "ghcup"
DEBUG Step "stack"
DEBUG Step "tlmgr"
DEBUG Step "myrepos"
DEBUG Step "chezmoi"
DEBUG Step "jetpack"
DEBUG Step "vim"
DEBUG Step "Neovim"
DEBUG Step "The Ultimate vimrc"
DEBUG Step "voom"
DEBUG Step "helix"
DEBUG Step "npm"
DEBUG Step "yarn"
DEBUG Step "pnpm"
DEBUG Step "deno"
DEBUG Step "composer"
DEBUG Step "krew"
DEBUG Step "helm"
DEBUG Step "gem"
DEBUG Detected "/opt/chefdk/embedded/bin/gem" as "gem"
DEBUG Step "rubygems"
DEBUG Step "julia"
DEBUG Step "haxelib"
DEBUG Step "sheldon"
DEBUG Step "stew"
DEBUG Step "rtcl"
DEBUG Step "bin"
DEBUG Step "gcloud"
DEBUG Detected "/usr/bin/gcloud" as "gcloud"

── 11:34:17 - gcloud ───────────────────────────────────────────────────────────
DEBUG Executing command `/usr/bin/gcloud components update --quiet`
Beginning update. This process may take several minutes.
ERROR: (gcloud.components.update)
You cannot perform this action because the Google Cloud CLI component manager
is disabled for this installation. You can run the following command
to achieve the same result for this installation:

sudo apt-get update && sudo apt-get --only-upgrade install google-cloud-sdk google-cloud-sdk-minikube google-cloud-sdk-app-engine-grpc google-cloud-sdk-cloud-build-local google-cloud-sdk-app-engine-python google-cloud-sdk-nomos google-cloud-sdk-config-connector google-cloud-sdk-app-engine-python-extras google-cloud-sdk-cloud-run-proxy google-cloud-sdk-spanner-migration-tool google-cloud-sdk-anthos-auth google-cloud-sdk-app-engine-java google-cloud-sdk-package-go-module google-cloud-sdk-app-engine-go google-cloud-sdk-harbourbridge google-cloud-sdk-cbt google-cloud-sdk-enterprise-certificate-proxy google-cloud-sdk-gke-gcloud-auth-plugin google-cloud-sdk-kpt google-cloud-sdk-bigtable-emulator google-cloud-sdk-pubsub-emulator google-cloud-sdk-terraform-tools google-cloud-sdk-log-streaming google-cloud-sdk-skaffold google-cloud-sdk-kubectl-oidc google-cloud-sdk-firestore-emulator google-cloud-sdk-datastore-emulator google-cloud-sdk-spanner-emulator kubectl google-cloud-sdk-local-extract

DEBUG Command failed: Err(
   0: Command failed: `/usr/bin/gcloud components update --quiet`
   1: `/usr/bin/gcloud` failed: exit status: 1

Location:
   src/steps/generic.rs:261)
DEBUG Step "gcloud" failed:
   0: Command failed: `/usr/bin/gcloud components update --quiet`
   1: `/usr/bin/gcloud` failed: exit status: 1

Location:
   src/steps/generic.rs:261
gcloud failed:
   0: Command failed: `/usr/bin/gcloud components update --quiet`
   1: `/usr/bin/gcloud` failed: exit status: 1

Location:
   src/steps/generic.rs:261
Retry? (y)es/(N)o/(s)hell/(q)uit
SteveLauC commented 1 year ago

581 should resolve this, the root cause of this issue is that our logger will ONLY be initialized after loading the configuration file, so you won't see any error logs until the loading is done, and the error that should be given under the case of this issue is just about the config file loading, so you got nothing

Tested on my host:

[misc]
ssume_yes = true

ssume_yes should be assume_yes, so topgrade will output:

$ ./target/debug/topgrade --dry-run --only system
ERROR Failed to deserialize /home/steve/.config/topgrade/topgrade.toml: unknown field `ssume_yes`, expected one of `pre_sudo`, `sudo_command`, `git_repos`, `predefined_git_repos`, `disable`, `ignore_failures`, `remote_topgrades`, `remote_topgrade_path`, `ssh_arguments`, `git_arguments`, `tmux_arguments`, `set_title`, `display_time`, `assume_yes`, `yay_arguments`, `aura_aur_arguments`, `aura_pacman_arguments`, `no_retry`, `run_in_tmux`, `cleanup`, `notify_each_step`, `accept_all_windows_updates`, `skip_notify`, `bashit_branch`, `only`, `no_self_update`, `log_filters` for key `misc` at line 6 column 1

── 11:17:24 - System update ────────────────────────────────────────────────────
Dry running: /usr/bin/sudo /usr/bin/dnf upgrade

── 11:17:24 - Summary ──────────────────────────────────────────────────────────
System update: OK
SteveLauC commented 1 year ago

I will release a new version of Topgrade recently, but if you don't want to wait for it, you can build the latest git version from the source instead.

One thing to note is that we will have a breaking change in this release, your git.arguments entry should be renamed to:

[git]
max_concurrency = 4
# Arguments to pass Git when pulling Repositories
pull_arguments = "--prune"

as Topgrade now can push and pull git repos

BTW, thanks for this detailed bug report!

daboross commented 1 year ago

and thanks for addressing this! cheers.