implement a dual-step / multi-step upgrade

[zerwes]

For example, if the current version is 22.7.11 and 23.1 is released, using opn_update_force=true will try to perform a direct upgrade to 23.1 ... and this will not work! So currently we need 2 ansible runs:

1. using -e opn_update_desired_version=22.7.11_1 (no reboot required)
2. followed by one using -e opn_update_desired_version=23.1 or -e opn_update_force=true

So as a default we should choose the next available release and using opn_update_force=true we should iterate over all available versions ...

[zerwes]

22.7.11_1 is not in the changelog fact ...

[zerwes]

# tail -11 /usr/local/opnsense/scripts/firmware/reboot.sh

# Additionally return the next version number if an update to the
# core package is available.  We want to use it to display additional
# information in the shell menu including the matching changelog.

if [ -n "${LQUERY}" -a -n "${RQUERY}" -a \
    "$(${PKG} version -t ${LQUERY} ${RQUERY})" = "<" ]; then
    echo ${RQUERY%%_*}
fi

# success is reboot:
exit ${WANT_REBOOT}

[zerwes]

next version working as expected:

# opnsense-version
OPNsense 23.1_6
# /usr/local/opnsense/scripts/firmware/reboot.sh
23.1.1

[zerwes]

question is: how will this behave if there is a major update in the pipe with a minor one between ...?

[zerwes]

Package name	Current version	New version	Required action	Repository
opnsense	23.1.1	23.1.1_2	upgrade	OPNsense

but

# /usr/local/opnsense/scripts/firmware/reboot.sh; echo $?
23.1.1
1

[zerwes]

-e opn_update_force=true works in this case as expected The following 1 package(s) will be affected (of 0 checked):\n\nInstalled packages to be UPGRADED:\n\topnsense: 23.1.1 -> 23.1.1_2 and after the update:

# opnsense-version 
OPNsense 23.1.1_2

[crpb]

i was wondering if it would be a good idea to make use of check.sh to detect the next safe upgrade-path so you could just use something like opn_auto_update=True instead of the opn_update_force which might result in issues when one of many systems is on a much older version than the rest.

this should also help with with an upgrade-till-latest mechanism.

here an quick overview

% for f in *.json ; do echo $f; cat $f |jq '{product_version, needs_reboot, upgrade_needs_reboot}, [.upgrade_sets[]|{name, new_version, current_version}], [.upgrade_packages[]|select(.name=="opnsense")|{new_version, current_version}] |del(select(length == 0))//empty?' ; done
pkg_upgrade.23.7.12_5.json
{
  "product_version": "23.7.12_5",
  "needs_reboot": "0",
  "upgrade_needs_reboot": "1"
}
[
  {
    "name": "packages",
    "new_version": "24.1",
    "current_version": "23.7"
  },
  {
    "name": "kernel",
    "new_version": "24.1",
    "current_version": "23.7.10"
  },
  {
    "name": "base",
    "new_version": "24.1",
    "current_version": "23.7.10"
  }
]
pkg_upgrade.23.7.5.json
{
  "product_version": "23.7.5",
  "needs_reboot": "1",
  "upgrade_needs_reboot": "0"
}
[
  {
    "new_version": "23.7.12_5",
    "current_version": "23.7.5"
  }
]
pkg_upgrade.24.1.1.json
{
  "product_version": "24.1.1",
  "needs_reboot": "1",
  "upgrade_needs_reboot": "0"
}
[
  {
    "new_version": "24.1.3_1",
    "current_version": "24.1.1"
  }
]
pkg_upgrade.24.1.2_1.json
{
  "product_version": "24.1.2_1",
  "needs_reboot": "0",
  "upgrade_needs_reboot": "0"
}
[
  {
    "new_version": "24.1.3_1",
    "current_version": "24.1.2_1"
  }
]
pkg_upgrade.24.1.3_1.json
{
  "product_version": "24.1.3_1",
  "needs_reboot": "0",
  "upgrade_needs_reboot": "0"
}

complete files: pkg_upgrade.23.7.5.json pkg_upgrade.23.7.12_5.json pkg_upgrade.24.1.1.json pkg_upgrade.24.1.2_1.json pkg_upgrade.24.1.3_1.json

[zerwes]

@crpb thank you for the hint I'll have a look at this option (but I must admit this topic is not tagged with a high priority for me at the moment ...)

[zerwes]

JSONFILE="/tmp/pkg_upgrade.json" TODO: install outdated version and test this

@crpb sorry, as I told you, this is not of high priority for us at the moment. so if you like to speed this up and give us a helping hand, pls create at least a draft as a PR ...

[crpb]

Hey, yeah no problem here. /me is currently very busy with work. And the comming week doesn't look either..

But I also wouldn't really know where to start. My Ansible Kung-Foo is very bad and that makes it harder to have a look when i had like 30minutes and was eager o_0. In that time nothing is done and more confusion is gathered *kicher*.

And yeah, for those couple json-dumps i had to do some loops and often took snapshots in virtsh and/or virt-manager.

[zerwes]

Hello @crpb Everybody is busy all day and has no time to play ... :-) No problem, if you can give me some hints about your steps to get the json files I can have a look at this. Just the commands you used might give me the hint into the right direction ... I assume it will be no big stuff to put this in ansible ...

[crpb]

if you can give me some hints about your steps

hmm.. when i see that correctly in my iso dump on the nas i ended up with 23.1 installed that and than it was basically this and now i see what's missing :-).

/usr/local/opnsense/scripts/firmware/check.sh;  cat /tmp/pkg_upgrade.json |jq '{product_version, needs_reboot, upgrade_needs_reboot}, [.upgrade_sets[]|{name, new_version, current_version}], [.upgrade_packages[]|select(.name=="opnsense")|{new_version, current_version}] |del(select(length == 0))//empty?' |tee -a /conf/`date -Iminutes`.json && /usr/local/opnsense/scripts/shell/firmware.sh

This looks quite better..

/usr/local/opnsense/scripts/firmware/check.sh;  cp /tmp/pkg_upgrade.json /conf/`date -Iminutes`.json && /usr/local/opnsense/scripts/shell/firmware.sh

EDIT: It was check.sh at the end but it must be firmware.sh :-). :EDIT

I extended it with autosaving. The second run of check.sh will prompt you, which is discarded by the piping, and you can just accept with 'y' or in some cases as you know with the next version-number.

I guess with that you would only have to keep in mind that you should take snapshots if something goes wrong. I certainly needed some because i forgot one and didn't realize it after two or three boot i think..

[crpb]

btw, /usr/local/opnsense/scripts/firmware/register.php resync for fixing those pkg resolve problems which came up a while ago.

[zerwes]

Thank you @crpb ; I'll have a look at this and will keep you updated ...

[zerwes]

current version: 24.1.9 community edition expected upgrade version: 24.4.1 business edition

# cat /tmp/pkg_upgrade.json | jq '{product_version, needs_reboot, upgrade_needs_reboot}, [.upgrade_sets[]|{name, new_version, current_version}], [.upgrade_packages[]|select(.name=="opnsense")|{new_version, current_version}] | del(select(length == 0))//empty?' | tee -a /conf/`date -Iminutes`.json
{
  "product_version": "24.1.9",
  "needs_reboot": "1",
  "upgrade_needs_reboot": "0"
}

[zerwes]

# cat /tmp/pkg_upgrade.json | jq '{product_version, needs_reboot, upgrade_needs_reboot}, [.new_packages[]|{name, version, current_version, new_version}], [.upgrade_packages[]|select(.name=="opnsense")|{new_version, current_version}]' 
{
  "product_version": "24.1.9",
  "needs_reboot": "1",
  "upgrade_needs_reboot": "0"
}
[
  {
    "name": "opnsense-business",
    "version": "24.4.1",
    "current_version": null,
    "new_version": null
  },
  {
    "name": "os-OPNBEcore",
    "version": "1.4_1",
    "current_version": null,
    "new_version": null
  }
]

[zerwes]

opnsense: 24.1.9 -> 24.1.9_4

cat /tmp/pkg_upgrade.json | jq '{product_version, needs_reboot, upgrade_needs_reboot}, [.upgrade_sets[]|{name, new_version, current_version}], [.upgrade_packages[]|select(.name=="opnsense")|{new_version, current_version}] | del(select(length == 0))//empty?'
{
  "product_version": "24.1.9",
  "needs_reboot": "0",
  "upgrade_needs_reboot": "0"
}
[
  {
    "new_version": "24.1.9_4",
    "current_version": "24.1.9"
  }
]

[crpb]

i wonder if those results should be implemented in the facts-script?

Basically to either run the upgrade in a loop(if thats possible) or at least i could script it localy e.g.

% ansible -m setup -o opnsense |sed 's/^.*=> {/{/g'|jq '[.ansible_facts.ansible_local.opnsense.core.product_version, .ansible_facts.ansible_hostname]|@tsv' -r | sort -V
24.1.8  fw-lan0
24.1.8  fw-lan0
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  gate
24.1.8  uniontel
24.1.9_4        gate
24.1.9_4        gate
24.1.9_4        gate
24.1.9_4        gate
24.1.9_4        gate
24.1.9_4        gate