Cannot upgrade to 23.7.7 without removing opnsense package

weebl2000 commented 1 year ago

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

[x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[x] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug

Latest update tries to replace "openssl" package with "openssl111" package and this somehow breaks the package manager logic.

To Reproduce

Steps to reproduce the behavior:

Try updating to 23.7.7

Expected behavior

Update happens

Describe alternatives you considered

-n/a

Screenshots

Relevant log files

***GOT REQUEST TO UPDATE***
Currently running OPNsense 23.7.7 at Wed Oct 25 10:42:55 CEST 2023
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Checking for upgrades (112 candidates): .......... done
Processing candidates (112 candidates): ..... done
Checking integrity... done (1 conflicting)
  - openssl111-1.1.1w conflicts with openssl-1.1.1w,1 on /usr/local/bin/c_rehash
Checking integrity... done (0 conflicting)
The following 53 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
    mpd5: 5.9_16
    openssl: 1.1.1w,1
    opnsense: 23.7.7

New packages to be INSTALLED:
    openssl111: 1.1.1w

Installed packages to be UPGRADED:
    curl: 8.3.0 -> 8.4.0
    easy-rsa: 3.1.6 -> 3.1.7
    gettext-runtime: 0.22_1 -> 0.22.3
    libnet: 1.2,1 -> 1.3,1
    libnghttp2: 1.56.0 -> 1.57.0
    lighttpd: 1.4.71 -> 1.4.72
    node_exporter: 1.5.0_7 -> 1.6.1
    nss: 3.93 -> 3.94
    opnsense-lang: 23.7.4 -> 23.7.7
    opnsense-update: 23.7.4 -> 23.7.7
    os-wireguard: 2.3 -> 2.4
    perl5: 5.34.1_3 -> 5.36.1_1
    py39-numpy: 1.25.0,1 -> 1.25.0_2,1
    py39-outcome: 1.2.0 -> 1.3.0
    suricata: 6.0.14 -> 6.0.15
    unbound: 1.18.0 -> 1.18.0_1
    vim: 9.0.1994 -> 9.0.2050
    xxd: 9.0.1994 -> 9.0.2050

Installed packages to be REINSTALLED:
    cpdup-1.22 (direct dependency changed: openssl111)
    cyrus-sasl-2.1.28 (direct dependency changed: openssl111)
    cyrus-sasl-gssapi-2.1.28 (direct dependency changed: openssl111)
    gmp-6.3.0 (option added: INFO)
    hostapd-2.10_8 (direct dependency changed: openssl111)
    hw-probe-1.6.5 (direct dependency changed: perl5)
    iperf3-3.15 (direct dependency changed: openssl111)
    isc-dhcp44-server-4.4.3P1 (direct dependency changed: openssl111)
    krb5-1.21.2 (direct dependency changed: openssl111)
    ldns-1.8.3 (direct dependency changed: openssl111)
    libevent-2.1.12 (direct dependency changed: openssl111)
    libfido2-1.13.0 (direct dependency changed: openssl111)
    liboping-1.8.0_2 (direct dependency changed: perl5)
    miniupnpd-2.3.3_1,1 (direct dependency changed: openssl111)
    monit-5.33.0 (direct dependency changed: openssl111)
    ntp-4.2.8p17_1 (direct dependency changed: openssl111)
    openldap26-client-2.6.6 (direct dependency changed: openssl111)
    openssh-portable-9.3.p2_1,1 (direct dependency changed: openssl111)
    openvpn-2.6.6 (direct dependency changed: openssl111)
    php82-8.2.11 (direct dependency changed: openssl111)
    pkcs11-helper-1.29.0 (direct dependency changed: openssl111)
    py39-aioquic-0.9.21 (direct dependency changed: openssl111)
    py39-cryptography-41.0.4,1 (direct dependency changed: openssl111)
    python39-3.9.18 (direct dependency changed: openssl111)
    rrdtool-1.8.0_2 (direct dependency changed: perl5)
    ruby-3.1.4_1,1 (direct dependency changed: openssl111)
    squid-5.9 (direct dependency changed: openssl111)
    strongswan-5.9.11_2 (direct dependency changed: openssl111)
    syslog-ng-4.4.0 (direct dependency changed: openssl111)
    tor-0.4.8.7 (direct dependency changed: openssl111)
    wpa_supplicant-2.10_9 (direct dependency changed: openssl111)

Number of packages to be removed: 3
Number of packages to be installed: 1
Number of packages to be upgraded: 18
Number of packages to be reinstalled: 31

The operation will free 22 MiB.
pkg-static: Cannot delete vital package: opnsense!
pkg-static: If you are sure you want to remove opnsense, 
pkg-static: unset the 'vital' flag with: pkg set -v 0 opnsense
Starting web GUI...done.
Generating RRD graphs...done.
***DONE***

Additional context

I could update to opnsense 23.7.7 manually by using pkg install, but it will still want to remove opnsense.

Environment

I was running 23.7.6

fichtner commented 1 year ago

It's in the release notes. You have third party packages installed.

weebl2000 commented 1 year ago

It's in the release notes. You have third party packages installed.

But why does this result in it trying to remove the opnsense package? I could understand if it would try to uninstall the third party packages. 🤔 It's trying to install openssl111, but then remove the opnsense package? Why would this happen if opnsense supports the openssl111 package.

fichtner commented 1 year ago

I don't have time for this. Please be patient. Everything will sort itself out when you give others the time to rebuild their repos that you've added to your install.

weebl2000 commented 1 year ago

I don't have time for this. Please be patient. Everything will sort itself out when you give others the time to rebuild their repos...

Yeah, no worries or hurries, I'm just confused is all. I'm only using packages installed from the opnsense web interface.

And I've managed to fix it by manually updating "pkg" to the latest version and then running the upgrade showed no conflicts. I guess somehow I had an outdated pkg version which couldn't handle the openssl -> openssl111 rename?

fichtner commented 1 year ago

No, updating pkg from FreeBSD (third party repo) has definitely made it worse.

# opnsense-revert pkg

weebl2000 commented 1 year ago

Okay, done. As you can obviously tell I have little experience in managing BSD systems. One final question and I will stop wasting your time: are the community plugins (like os-tor) considered "third-party"?

fkonradmain commented 1 year ago

I don't know if it helps with debugging, but I am running into the same issue with a similar set of packages, trying to update from Version 23.7.6:

***GOT REQUEST TO UPDATE***
Currently running OPNsense 23.7.6 at Wed Oct 25 09:30:14 UTC 2023
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Checking for upgrades (120 candidates): .......... done
Processing candidates (120 candidates): ..... done
Checking integrity... done (1 conflicting)
  - openssl111-1.1.1w conflicts with openssl-1.1.1w,1 on /usr/local/bin/c_rehash
Cannot solve problem using SAT solver, trying another plan
Checking integrity... done (0 conflicting)
The following 61 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
    mpd5: 5.9_16
    openssl: 1.1.1w,1
    opnsense: 23.7.6

New packages to be INSTALLED:
    openssl111: 1.1.1w

Installed packages to be UPGRADED:
    curl: 8.3.0 -> 8.4.0
    easy-rsa: 3.1.6 -> 3.1.7
    gettext-runtime: 0.22_1 -> 0.22.3
    libnet: 1.2,1 -> 1.3,1
    libnghttp2: 1.56.0 -> 1.57.0
    lighttpd: 1.4.71 -> 1.4.72
    node_exporter: 1.5.0_7 -> 1.6.1
    nss: 3.93 -> 3.94
    opnsense-lang: 23.7.4 -> 23.7.7
    opnsense-update: 23.7.4 -> 23.7.7
    os-ddclient: 1.16 -> 1.16_1
    p11-kit: 0.24.1_2 -> 0.25.0
    perl5: 5.34.1_3 -> 5.36.1_1
    py39-boto3: 1.28.62 -> 1.28.63
    py39-botocore: 1.31.62 -> 1.31.63
    py39-numpy: 1.25.0,1 -> 1.25.0_2,1
    py39-outcome: 1.2.0 -> 1.3.0
    suricata: 6.0.14 -> 6.0.15
    unbound: 1.18.0 -> 1.18.0_1

Installed packages to be REINSTALLED:
    cpdup-1.22 (direct dependency changed: openssl111)
    cyrus-sasl-2.1.28 (direct dependency changed: openssl111)
    cyrus-sasl-gssapi-2.1.28 (direct dependency changed: openssl111)
    ddclient-devel-3.10.0_7 (direct dependency changed: perl5)
    gmp-6.3.0 (option added: INFO)
    hostapd-2.10_8 (direct dependency changed: openssl111)
    iperf3-3.15 (direct dependency changed: openssl111)
    isc-dhcp44-server-4.4.3P1 (direct dependency changed: openssl111)
    krb5-1.21.2 (direct dependency changed: openssl111)
    ldns-1.8.3 (direct dependency changed: openssl111)
    libevent-2.1.12 (direct dependency changed: openssl111)
    libfido2-1.13.0 (direct dependency changed: openssl111)
    lldpd-1.0.14 (direct dependency changed: openssl111)
    monit-5.33.0 (direct dependency changed: openssl111)
    net-snmp-5.9.1_4,1 (direct dependency changed: openssl111)
    ntp-4.2.8p17_1 (direct dependency changed: openssl111)
    openldap26-client-2.6.6 (direct dependency changed: openssl111)
    openssh-portable-9.3.p2_1,1 (direct dependency changed: openssl111)
    openvpn-2.6.6 (direct dependency changed: openssl111)
    p5-Data-Validate-IP-0.27 (direct dependency changed: perl5)
    p5-IO-Socket-IP-0.42 (direct dependency changed: perl5)
    p5-IO-Socket-SSL-2.083_1 (direct dependency changed: perl5)
    p5-Mozilla-CA-20230821 (direct dependency changed: perl5)
    p5-Net-SSLeay-1.92 (direct dependency changed: openssl111)
    p5-NetAddr-IP-4.079 (direct dependency changed: perl5)
    php82-8.2.11 (direct dependency changed: openssl111)
    pkcs11-helper-1.29.0 (direct dependency changed: openssl111)
    pkg-1.19.2
    py39-aioquic-0.9.21 (direct dependency changed: openssl111)
    py39-cryptography-41.0.4,1 (direct dependency changed: openssl111)
    python39-3.9.18 (direct dependency changed: openssl111)
    rrdtool-1.8.0_2 (direct dependency changed: perl5)
    ruby-3.1.4_1,1 (direct dependency changed: openssl111)
    squid-5.9 (direct dependency changed: openssl111)
    strongswan-5.9.11_2 (direct dependency changed: openssl111)
    syslog-ng-4.4.0 (direct dependency changed: openssl111)
    tor-0.4.8.7 (direct dependency changed: openssl111)
    wpa_supplicant-2.10_9 (direct dependency changed: openssl111)

Number of packages to be removed: 3
Number of packages to be installed: 1
Number of packages to be upgraded: 19
Number of packages to be reinstalled: 38

The operation will free 22 MiB.
pkg-static: Cannot delete vital package: opnsense!
pkg-static: If you are sure you want to remove opnsense, 
pkg-static: unset the 'vital' flag with: pkg set -v 0 opnsense
Starting web GUI...done.
Generating RRD graphs...done.
***DONE***

Unlike the original post, my system is not trying to patch:

< os-wireguard: 2.3 -> 2.4
< vim: 9.0.1994 -> 9.0.2050
< xxd: 9.0.1994 -> 9.0.2050

Yet it is trying to patch the following other packages:

> os-ddclient: 1.16 -> 1.16_1
> p11-kit: 0.24.1_2 -> 0.25.0
> py39-boto3: 1.28.62 -> 1.28.63
> py39-botocore: 1.31.62 -> 1.31.63

If it doesn't help, I'll be happy to delete this comment.

weebl2000 commented 1 year ago

If it doesn't help, I'll be happy to delete this comment.

Somehow by temporarily using the FreeBSD pkg and then updating the packages fixed all issues for me. And all packages are back to the OPNSense source now.

I don't know the impact of this so move cautiously and only do it if you know more about FreeBSD than I do.

Only extra packages I have are:

fichtner commented 1 year ago

Official plugins like os-tor are not third party and work fine on 23.7.7 itself, provided the openssl111 update was properly installed.

Let’s give third party repo maintainers the time to update their packages. This is just a transient issue.

saltyzip commented 1 year ago

Same issue here and I thought I was pretty much using a stock build:

GOT REQUEST TO UPDATE Currently running OPNsense 24.1.a_361 at Wed Oct 25 11:14:26 BST 2023 Updating OPNsense repository catalogue... OPNsense repository is up to date. All repositories are up to date. Updating OPNsense repository catalogue... OPNsense repository is up to date. All repositories are up to date. Checking for upgrades (115 candidates): .......... done Processing candidates (115 candidates): ..... done Checking integrity... done (1 conflicting)

openssl111-1.1.1w conflicts with openssl-1.1.1w,1 on /usr/local/bin/c_rehash Cannot solve problem using SAT solver, trying another plan Checking integrity... done (0 conflicting) The following 56 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED: mpd5: 5.9_16 openssl: 1.1.1w,1 opnsense-devel: 24.1.a_361

New packages to be INSTALLED: openssl111: 1.1.1w

Installed packages to be UPGRADED: curl: 8.3.0 -> 8.4.0 easy-rsa: 3.1.6 -> 3.1.7 gettext-runtime: 0.22_1 -> 0.22.3 libnet: 1.2,1 -> 1.3,1 libnghttp2: 1.56.0 -> 1.57.0 lighttpd: 1.4.71 -> 1.4.72 nss: 3.93 -> 3.94 opnsense-lang: 23.7.4 -> 23.7.7 opnsense-update: 23.7.4 -> 23.7.7 os-ddclient: 1.16 -> 1.16_1 perl5: 5.34.1_3 -> 5.36.1_1 py39-boto3: 1.28.62 -> 1.28.63 py39-botocore: 1.31.62 -> 1.31.63 py39-dns-lexicon: 3.15.0 -> 3.16.0 py39-numpy: 1.25.0,1 -> 1.25.0_2,1 py39-outcome: 1.2.0 -> 1.3.0 suricata-devel: 7.0.1 -> 7.0.2 unbound: 1.18.0 -> 1.18.0_1

Installed packages to be REINSTALLED: bind-tools-9.18.19 (direct dependency changed: openssl111) cpdup-1.22 (direct dependency changed: openssl111) cyrus-sasl-2.1.28 (direct dependency changed: openssl111) cyrus-sasl-gssapi-2.1.28 (direct dependency changed: openssl111) ddclient-devel-3.10.0_7 (direct dependency changed: perl5) gmp-6.3.0 (option added: INFO) hostapd-2.10_8 (direct dependency changed: openssl111) isc-dhcp44-server-4.4.3P1 (direct dependency changed: openssl111) krb5-1.21.2 (direct dependency changed: openssl111) ldns-1.8.3 (direct dependency changed: openssl111) libevent-2.1.12 (direct dependency changed: openssl111) libfido2-1.13.0 (direct dependency changed: openssl111) monit-5.33.0 (direct dependency changed: openssl111) ntp-4.2.8p17_1 (direct dependency changed: openssl111) openldap26-client-2.6.6 (direct dependency changed: openssl111) openssh-portable-9.3.p2_1,1 (direct dependency changed: openssl111) openvpn-2.6.6 (direct dependency changed: openssl111) p5-Data-Validate-IP-0.27 (direct dependency changed: perl5) p5-IO-Socket-IP-0.42 (direct dependency changed: perl5) p5-IO-Socket-SSL-2.083_1 (direct dependency changed: perl5) p5-Mozilla-CA-20230821 (direct dependency changed: perl5) p5-Net-SSLeay-1.92 (direct dependency changed: openssl111) p5-NetAddr-IP-4.079 (direct dependency changed: perl5) php82-8.2.11 (direct dependency changed: openssl111) pkcs11-helper-1.29.0 (direct dependency changed: openssl111) py39-aioquic-0.9.21 (direct dependency changed: openssl111) py39-cryptography-41.0.4,1 (direct dependency changed: openssl111) python39-3.9.18 (direct dependency changed: openssl111) rrdtool-1.8.0_2 (direct dependency changed: perl5) socat-1.7.4.4 (direct dependency changed: openssl111) squid-5.9 (direct dependency changed: openssl111) strongswan-5.9.11_2 (direct dependency changed: openssl111) syslog-ng-4.4.0 (direct dependency changed: openssl111) wpa_supplicant-2.10_9 (direct dependency changed: openssl111)

Number of packages to be removed: 3 Number of packages to be installed: 1 Number of packages to be upgraded: 18 Number of packages to be reinstalled: 34

The operation will free 22 MiB. pkg-static: Cannot delete vital package: opnsense-devel! pkg-static: If you are sure you want to remove opnsense-devel, pkg-static: unset the 'vital' flag with: pkg set -v 0 opnsense-devel Starting web GUI...done. Generating RRD graphs...done. DONE

jochristian commented 1 year ago

Same issue Tried a completely new installation now. Downloaded the latest iso from opnsense. Installed and tried to upgrade.

Getting the same issue. Clean installation.

drewzh commented 1 year ago

Same here for what it's worth (sorry for adding more noise). Stock install - can't update with similar issues relating to openssl111.

michamos commented 1 year ago

Same issue for me without any packages from 3rd party repositories afaik.

Interestingly, the update wants to remove mpd5, which opnsense depends on. Might mpd5 still depend on the old openssl package rather than openssl111? For my current 23.7.6 system, I see the following deps:

# pkg info -d mpd5
mpd5-5.9_16
        openssl-1.1.1w,1

I don't know how to check the dependencies in the update without applying it.

AdSchellevis commented 1 year ago

Can you try the following?

pkg install -f mpd5

It seems that I have a test machine with a similar problem, which did pull itself through the upgrade after forcing a reinstall of mpd5

FabianMartin commented 1 year ago

Can you try the following?
pkg install -f mpd5
It seems that I have a test machine with a similar problem, which did pull itself through the upgrade after forcing a reinstall of mpd5

Worked here. New installation updated successfully.

Randynho commented 1 year ago

jochristian

Same with me. Could have spared the fresh install

Randynho commented 1 year ago

Can you try the following?
pkg install -f mpd5
It seems that I have a test machine with a similar problem, which did pull itself through the upgrade after forcing a reinstall of mpd5

This worked for me! Thanks a lot!

fichtner commented 1 year ago

I can add a workaround for mpd5, but there is really nothing special about mpd5 and it didn't cause an issue in all the update tests I did from various 23.7.x versions.

tensai44 commented 1 year ago

Can you try the following?
pkg install -f mpd5
It seems that I have a test machine with a similar problem, which did pull itself through the upgrade after forcing a reinstall of mpd5

Worked for me too!

fichtner commented 1 year ago

https://github.com/opnsense/ports/commit/22f51cb9

Hotfixed...

fichtner commented 1 year ago

Reported this upstream here https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273970 and solved for us so closing.

johnserri commented 1 year ago

Can you try the following?
pkg install -f mpd5
It seems that I have a test machine with a similar problem, which did pull itself through the upgrade after forcing a reinstall of mpd5

thanks you for your reply and , hey every one this fix the issue ..

opnsense / core

Cannot upgrade to 23.7.7 without removing opnsense package #6959