search

helloSystem / hello

Desktop system for creators with a focus on simplicity, elegance, and usability. Based on FreeBSD. Less, but better!
2.3k stars 57 forks source link

Thoughts on the FreeBSD ports/package system #161

Open probonopd opened 3 years ago

probonopd commented 3 years ago

While we have been working with FreeBSD packages for a while, it seems like the structure of how packages get built and distributed leaves a few aspects to be deserved. This issue describes our gripes in the hope that someone can help us find solutions.

grahamperrin commented 2 years ago

I don't know this behavior from Ubuntu/Debian.

Apples, oranges; not a meaningful comparison.

The ports collection for FreeBSD OS is not Ubuntu.

reddit.com/r/LinusTechTips/comments/qrvmq0/linus_uninstalling_his_desktop_environment_in_the

Deleted/shitpost

probonopd commented 2 years ago

Deleted/shitpost

The article basically described well-known YouTuber Linus Sebastian inadvertently uninstalling the desktop environment while trying to install something. Linus is not exactly a noob when it comes to computers. But who reads all the verbose gibberish package managers throw at you... Generated quite an uproar back at the time.

grahamperrin commented 2 years ago

graphics/drm-510-kmod

https://github.com/helloSystem/ISO/issues/342#issuecomment-1236394769

I think they start building against the latest released minor version of FreeBSD once the previous one is eol,

13.0 is end of life: https://www.freebsd.org/security/unsupported/.

a quarter after the new minor version came out. And then we have to wait (in the worst case) one quarter for it to appear in quarterly. Or am I wrong?

From https://github.com/helloSystem/ISO/issues/342#issuecomment-1236385303:

Visually:

image

– that is, a run of poudriere for quarterly

If successful, this run might lead to population of the quarterly repo for FreeBSD:13:amd64.

I don't imagine waiting three months (a quarter).

(is this documented somewhere clearly?)

Probably worth noting the IGNORE context at https://github.com/freebsd/freebsd-ports/commit/7377fed7a063420992359af3a15006fa4fee6303#diff-3fcb57d0457b20a555b9c0793151e547561c8def0092164c3c5a22030f127c88R25-R27 – the run of poudriere is for 13.1, so graphics/drm-510-kmod is not ignored.

More generally …

Policies et cetera

grahamperrin commented 2 years ago

https://github.com/helloSystem/Utilities/issues/131#issuecomment-1236386756

… an example of one port superseding another.

https://github.com/helloSystem/Utilities/issues/131#issuecomment-1236388162

… What was installed (at the ISO creation time) should never be uninstalled, unless explicitly asked for by the user. But it should be possible to be updated. …

Recall Python. A latest example (without helloSystem): 

…
Processing candidates (173 candidates): 100%
Checking integrity... done (5 conflicting)
  - libxcvt-0.1.2_1 conflicts with xorg-server-1.20.13,1 on /usr/local/bin/cvt
  - py39-cairo-1.21.0,1 conflicts with py38-cairo-1.18.1_2,1 on /usr/local/include/pycairo/py3cairo.h
  - py39-markdown-3.3.7 conflicts with py38-markdown-3.3.4 on /usr/local/bin/markdown_py
  - openldap26-client-2.6.3 conflicts with openldap24-client-2.4.59_4 on /usr/local/bin/ldapadd
  - openldap26-client-2.6.3 conflicts with openldap24-client-2.4.59_4 on /usr/local/bin/ldapadd
Checking integrity... done (0 conflicting)
The following 199 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
    openldap24-client: 2.4.59_4
    py38-cairo: 1.18.1_2,1
    py38-gobject3: 3.38.0
    py38-markdown: 3.3.4

New packages to be INSTALLED:
    Imath: 3.1.5_1
    highway: 1.0.1
    libdeflate: 1.13
    libimagequant: 2.17.0
    libjxl: 0.7.r
    libsoup3: 3.1.1
    libxcvt: 0.1.2_1
    openexr: 3.1.5
    openldap26-client: 2.6.3
    py39-cairo: 1.21.0,1
    py39-dnspython: 2.2.1_1,1
    py39-evdev: 1.6.0
    py39-gobject3: 3.42.2
    py39-importlib-metadata: 4.8.1
    py39-markdown: 3.3.7
    py39-pyudev: 0.22.0
    py39-setuptools: 63.1.0
    py39-six: 1.16.0
    py39-zipp: 3.4.0
    python39: 3.9.13
    spidermonkey91: 91.12.0_1
    svt-av1: 1.2.1

Installed packages to be UPGRADED:
…

Re: https://github.com/helloSystem/ISO/tree/experimental/settings I see, for example, https://github.com/helloSystem/ISO/commit/2491528e832bb38842312d1d57b18f8a75bd8b05#diff-ba48011d35001eeaa41235c7a115fc5007325aacc87366e0858bf10e3c5fb456R77 devel/py-pytz in packages.hello. It's also a runtime requirement for dozens of other ports: https://www.freshports.org/devel/py-pytz/#dependencies.

It's good that you no longer hardcode the version e.g. py37-pytz, but (for Python generally) it's good to be keep abreast of changes, for example https://redd.it/vm3tkh recent 3.9 by default. Was what's integral to helloSystem prepared for that change?

grahamperrin commented 2 years ago

NB https://github.com/helloSystem/ISO/issues/55#issuecomment-1236480409 re: helloDesktop.

grahamperrin commented 2 years ago

https://github.com/helloSystem/ISO/issues/396#issuecomment-1236599922 @darkoverlordofdata wrote:

… sudo pkg upgrade and type yes to all prompts. …

More than one prompt may be expected when, for example, an upgrade to pkg is available.

https://www.freshports.org/ports-mgmt/pkg/#history

Sometimes, it's necessary to run pkg-upgrade(8) more than once, because a single run can not complete everything that's required for coherence. https://github.com/helloSystem/hello/issues/161#issuecomment-1236469260 above, IIRC the OpenLDAP client was a cause of the end of an initial run (the run that preceded what's above). NB the CONFLICTS_INSTALL lines in https://www.freshports.org/net/openldap24-client/ and https://www.freshports.org/net/openldap26-client/.

https://github.com/freebsd/freebsd-ports/blob/8a90098e6b19316d1403ff8ba109a188df369036/UPDATING#L8-L19 was a very rare example of a requirement to use pkg-delete(8) to --force removal of a single package prior to an upgrade.

Generally

It's good practice for all users of ports (and packages of ports) to at least be aware of UPDATING i.e. https://cgit.freebsd.org/ports/tree/UPDATING. We don't expect all readers to grasp the technical aspects, however it can be a useful point of reference when queries do arise.

In addition, for developers

CHANGES https://cgit.freebsd.org/ports/tree/CHANGES.

Postscript

Yeah, here's the OpenLDAP client conflict on another machine:

image

grahamperrin commented 2 years ago

http://beefy14.nyi.freebsd.org/build.html?mastername=131amd64-quarterly&build=8327a5f1bca6 queued for quarterly.

drm-510-kmod-5.10.113_1 succeeded. Head and tail of the log: 

=>> Building graphics/drm-510-kmod
build started at Mon Sep  5 14:49:39 UTC 2022
port directory: /usr/ports/graphics/drm-510-kmod
package name: drm-510-kmod-5.10.113_1
building for: FreeBSD 131amd64-quarterly-job-08 13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 amd64
maintained by: x11@FreeBSD.org
Makefile ident: 
Poudriere version: 3.2.8-21-g883afb07
Host OSVERSION: 1400063
Jail OSVERSION: 1301000
Job Id: 08

---Begin Environment---
SHELL=/bin/csh
OSVERSION=1301000
UNAME_v=FreeBSD 13.1-RELEASE-p2
UNAME_r=13.1-RELEASE-p2
…
build of graphics/drm-510-kmod | drm-510-kmod-5.10.113_1 ended at Mon Sep  5 14:55:24 UTC 2022
build time: 00:05:45

39 ports failed; 8,395 of 32,238 ports remain:

image

probonopd commented 2 years ago

Currently, we are just installing the packages at ISO creation time.

Unfortunately, pkg may suggest to the user to REMOVE some of those packages.

We cannot lock all of them because that would prevent them from being updated.

Our objective is that all packages that are installed at ISO creation time are considered to be "part of the system" and won't be inadvertently REMOVED by pkg later on when the user is using the system.

Maybe the proper way would be to not just install them at ISO creation time, but define them as dependencies of the hello package, and then locking the hello package. Wouldn't that require the dependencies of the hello package to remain installed on the system no matter what?

Note to self: A script like this can be used to convert the packages.* files in https://github.com/helloSystem/ISO/tree/experimental/settings to dependencies that can be used in pkg manifest files in https://github.com/helloSystem/ISO/blob/experimental/overlays/uzip/:

#!/bin/sh

# Also see: https://github.com/pageflt/freebsd-mkmnfst

wget -c "https://raw.githubusercontent.com/helloSystem/ISO/da8bc8d510bb56b66ba7bae7c5ffc8a269da10ab/settings/packages.hello" -O /tmp/list

# Returns an entry suitable for pkg '+MANIFEST' files
echo_manifest_line()
{
  # Get the version of the package
  V=$(pkg rquery %v "${1}" | xargs) 
  # Get the origin (e.g., 'shells/bash') of the package
  O=$(pkg rquery %o "${1}" | xargs) 
  # Get the name of the package
  N=$(pkg rquery %n "${1}" | xargs)  
  MANIFEST_LINE="    ${N}: {origin: ${O}, version: 0},"
  if [ "${V}" == "" ] ; then
    echo "!!! No match for" $line
  else
    echo "${MANIFEST_LINE}"
  fi
}

grep -v '^#' /tmp/list | grep -v '^http' | grep -v '^cirrus' | sort | uniq | while read -r line ; do
  echo_manifest_line $line
done

With this approach I don't get dependency lines for

!!! No match for py37-beautifulsoup
!!! No match for py37-dateutil
!!! No match for py37-psutil
!!! No match for py37-pytz
!!! No match for py37-qt5-dbus
!!! No match for py37-qt5-multimedia
!!! No match for py37-qt5-network
!!! No match for py37-qt5-qml
!!! No match for py37-qt5-webengine
!!! No match for py37-qt5-widgets
!!! No match for py37-xdg
!!! No match for py37-xmltodict
!!! No match for ssvnc
!!! No match for xf86-video-qxl # !i386

How can one specify, in a manifest file, "whatever Python version is currently being used"?

probonopd commented 2 years ago

Also, right now a mess of different Python versions gets installed.

    py27-setuptools44: 44.1.1
    py27-sqlite3: 2.7.18_7
    py310-setuptools: 62.1.0_1
    py310-sqlite3: 3.10.7_7
    py311-setuptools: 62.1.0_1
    py311-sqlite3: 3.11.0.r1_7
    py37-setuptools: 62.1.0_1
    py37-sqlite3: 3.7.14_7
    py38-setuptools: 62.1.0_1
    py38-sqlite3: 3.8.14_7
    py39-beautifulsoup: 4.11.1
    py39-brotli: 1.0.9
    py39-cairo: 1.18.1_2,1
    py39-certifi: 2022.5.18.1
    py39-cffi: 1.15.0_1
    py39-charset-normalizer: 2.0.12
    py39-cryptography: 3.4.8
    py39-dateutil: 2.8.2
    py39-distro: 1.7.0
    py39-dnspython: 2.2.1,1
    py39-gobject3: 3.38.0_1
    py39-html5lib: 1.0.1
    py39-idna: 3.3
    py39-importlib-metadata: 4.8.1
    py39-lxml: 4.9.0
    py39-markdown: 3.3.7
    py39-mutagen: 1.45.1
    py39-olefile: 0.46
    py39-openssl: 20.0.1,1
    py39-pillow: 9.1.1
    py39-pip: 22.1.2
    py39-psutil: 5.9.1_2
    py39-pycparser: 2.21
    py39-pycryptodomex: 3.12.0
    py39-pycups: 2.0.1
    py39-pycurl: 7.45.1
    py39-pyelftools: 0.27
    py39-pysocks: 1.7.1
    py39-pytz: 2021.3,1
    py39-qt5-webengine: 5.15.5_3
    py39-requests: 2.28.0
    py39-soupsieve: 2.0.1
    py39-sqlite3: 3.9.13_7
    py39-tkinter: 3.9.13_6
    py39-urllib3: 1.26.9,1
    py39-webencodings: 0.5.1
    py39-websockets: 10.3
    py39-xdg: 0.28
    py39-xmltodict: 0.13.0
    py39-zipp: 3.4.0
    pygobject3-common: 3.38.0_1
    python27: 2.7.18_2
    python3: 3_3
    python310: 3.10.7
    python311: 3.11.0.r1
    python37: 3.7.14
    python38: 3.8.14

How can we tell the system to use one, and just one, Python version for everything?

probonopd commented 2 years ago

Related:

Say we need PyQt as part of helloSystem, and it must never be uninstalled. However, as the user updates the system, the main Python version may change.

How do we specify this?

probonopd commented 2 years ago

Maybe the proper way would be to not just install them at ISO creation time, but define them as dependencies of the hello package

With https://github.com/helloSystem/ISO/blob/dc750566ebc565937553bf7ea76b8b6a03b3a214/overlays/uzip/hello/manifest#L23-L108 the pkg gets created, but trying to install it fails with

Installing hello-0.8.0_0H80...
Failed to install the following 1 package(s): /usr/local/furybsd/amd64/cache/packages/transient/hello-0.8.0_0H80.pkg

https://cirrus-ci.com/task/4797302459072512?logs=Build#L1800

Why?

Possibly because pkg-static can't handle the dependencies? This is how we are calling it:

https://github.com/helloSystem/ISO/blob/aa8d54928a54660f8ee16696e09e343963035443/build.sh#L227

How to do this properly?

Using pkg instead of pkg-static gives a more verbose error, it complains about Missing dependency 'archivemount' rather than installing it. Why? How can we tell pkg to install dependencies rather than to complain about them?

+ /usr/local/sbin/pkg -r /usr/local/furybsd/uzip add /usr/local/furybsd/amd64/cache/packages/transient/hello-0.8.0_0H85.pkg
Installing hello-0.8.0_0H85...
pkg: Missing dependency 'archivemount'
Failed to install the following 1 package(s): /usr/local/furybsd/amd64/cache/packages/transient/hello-0.8.0_0H85.pkg
grahamperrin commented 2 years ago

... and then locking the hello package. ...

Locking a package affects that package alone.

grahamperrin commented 2 years ago

https://github.com/helloSystem/hello/issues/161#issuecomment-1242680531 and so on, having so many diverse overlapping questions in a single issue becomes difficult to follow, GitHub issues don't lend themselves to this type of thing.

Instead, try asking questions as Q&A in discussions, https://github.com/helloSystem/hello/discussions/

grahamperrin commented 2 years ago

cirrus-ci.com/task/4797302459072512?logs=Build#L1800

Why is something for bhnd_erom(9) highlighted?

probonopd commented 2 years ago

Locking a package affects that package alone.

Yes, but since the locked package depends on the packages we want to prevent from being removed, those packages won't be removed since that would also require the locked package to be removed. Which the lock prevents. At least that is the theory...

grahamperrin commented 2 years ago

since the locked package depends on the packages we want to prevent from being removed, those packages won't be removed since that would also require the locked package to be removed. Which the lock prevents. At least that is the theory...

With these packages locked by default (a fresh installation of helloSystem 0.7.0):

-- at a glance, these potential removals:

Context: 0.7.0 ea31abc261f 2022-09-11 typescript 01.txt

Postscript

I lost sight of this:

dependencies of the hello package, and then locking the hello package.

grahamperrin commented 2 years ago

... And then we have to wait (in the worst case) one quarter for it to appear in quarterly. Or am I wrong? ...

Not so long. It appeared earlier today:

image

grahamperrin commented 2 years ago

https://github.com/helloSystem/ISO/issues/51 | https://github.com/helloSystem/hello/discussions/272#discussioncomment-3632245 etc.

probonopd commented 1 year ago

https://github.com/helloSystem/ISO/issues/51 | https://github.com/helloSystem/hello/discussions/272#discussioncomment-3632245 etc.

@grahamperrin if you are using links, if you are doing it like this, then they get displayed nicely with titles (much more readable!) in GitHub automatically:

References:
* https://github.com/helloSystem/ISO/issues/51
* https://github.com/helloSystem/hello/discussions/272#discussioncomment-3632245

References:

probonopd commented 1 year ago

Maybe the proper way would be to not just install them at ISO creation time, but define them as dependencies of the hello package

My attempts at implementing https://github.com/helloSystem/hello/issues/161#issuecomment-1242682479 were not yet successful and in fact broke the ISO build.

I tried to

but apparently pkg add cannot install dependencies whereas pkg install requires the pkg to be installed to be in a repository, which we don't have.

were unsuccesful attempts at working around at this.

Maybe @crees has an idea how to do this best. Do we need something like

?

grahamperrin commented 1 year ago

… using links, if you are doing it like this, then they get displayed nicely with titles (much more readable!) in GitHub automatically: …

Apparently, no automated presentation of titles for the three commits and one PR here:

image

I'm happy with pointing to reveal context, for example:

image

Beyond that: https://github.com/community/ is probably a better place to discuss.

probonopd commented 1 year ago

Apparently, no automated presentation of titles for the three commits and one PR here

Seemingly doesn't work for commits.

probonopd commented 1 year ago

Maybe a better approach for https://github.com/helloSystem/hello/issues/161#issuecomment-1242674360:

The following script should be run when the freshly installed system is running for the first time, or at the end of helloSystem ISO creation. Its purpose is to create a package that depends on all non-automatic packages on this system, so that we can lock this package in the hope that pkg will never uninstall (but may update) those packages during future updates and upgrades of the installed system.

Note that we would like to use only origin instead of package names in the hope that this would handle minor Python version updates more gracefully (we don't want to depend on any particular Pyhton version because these tend to change all the time). E.g., when a pkg upgrade wants to replace py37-... with py38-... or py39-... then we want to allow that. For example, we would want to depend on x11-toolkits/py-qt5-widgets but not on py38-qt5-widgets specifically, so that once py39 is there the user can seamlessly upgrade to it. How can this be achieved? The FreeBSD way would probably be to have a package repository in which there is a package the dependencies of which get updated to always match e.g., the Python version that is in the FreeBSD package repository at that point in time. However, helloSystem does not want to operate a package repository because it would require infrastructure to be maintained. So possibly some additional trickery is needed, e.g., having a local FreeBSD package repository that would contain a package which has the required dependency with the then (at the time of the update/upgrade) used Python versions,... or somehow unlocking, uninstalling, rewriting, installing, locking this package on-the-fly?

#!/bin/sh

# This script should be run when the freshly installed system is running for the first time,
# or at the end of helloSystem ISO creation. Its purpose is to create a package that
# depends on all non-automatic packages on this system, so that we can lock this package
# in the hope that pkg will never uninstall (but may update) those packages during future 
# updates and upgrades of the installed system.

# Intended result e.g.,
# % sudo pkg remove -y mountarchive               
# Updating database digests format: 100%
# pkg: $PACKAGE_NAME is locked, cannot delete mountarchive
# Cannot perform request

PACKAGE_VERSION=0
PACKAGE_NAME=hellosystem-essential-packages

# Unlock and uninstall previously installed package, if available
sudo pkg unlock -y $PACKAGE_NAME 2>/dev/null || true
sudo pkg remove -y $PACKAGE_NAME 2>/dev/null || true

# List non-automatic packages
# Note that we would like to use only origin instead of package names
# in the hope that this would handle minor Python version updates more gracefully
# (we don't want to depend on any particular Pyhton version because these
# tend to change all the time)
DEP_LINES=$(pkg query -e '%a =     0' '"%n": {origin: "%o", version: "%v"},')

echo $DEP_LINES

# Create manifest
# https://hackmd.io/@dch/HkwIhv6x7

cat > /tmp/manifest.ucl <<EOF
name:         $PACKAGE_NAME
origin:       hellosystem/$PACKAGE_NAME
comment:      "Packages on the helloSystem ISO."
prefix:       /usr/local
version:      $PACKAGE_VERSION
www:          ""
maintainer:   ""
deps:         {
$DEP_LINES
              }
desc:         <<EOD
This package depends on all non-automatic packages on the helloSystem ISO.
Locking this package should prevent packages that come with the helloSystem ISO
from being inadvertently removed.
EOD
EOF

# Create package

pkg create --verbose --root-dir /var/empty --manifest /tmp/manifest.ucl --out-dir /tmp/

# Install package. Note that 'pkg add' only works if all dependencies have already
# been installed on the system, which is the case here

sudo pkg add /tmp/$PACKAGE_NAME-$PACKAGE_VERSION.pkg

pkg info $PACKAGE_NAME
echo -n "Dependencies: "
pkg info -dx $PACKAGE_NAME

sudo pkg lock -y $PACKAGE_NAME
probonopd commented 1 year ago

Currently this ends up with dependencies like

"py38-beautifulsoup": {origin: "www/py-beautifulsoup", version: "4.10.0"},
"py38-dateutil": {origin: "devel/py-dateutil", version: "2.8.1"},
"py38-pip": {origin: "devel/py-pip", version: "20.3.4"},
"py38-psutil": {origin: "sysutils/py-psutil", version: "5.8.0"},
"py38-pytz": {origin: "devel/py-pytz", version: "2021.1,1"},
"py38-qt5-dbus": {origin: "devel/py-qt5-dbus", version: "5.15.4_2"},
"py38-qt5-multimedia": {origin: "multimedia/py-qt5-multimedia", version: "5.15.4_3"},
"py38-qt5-network": {origin: "net/py-qt5-network", version: "5.15.4_2"},
"py38-qt5-qml": {origin: "lang/py-qt5-qml", version: "5.15.4_3"},
"py38-qt5-webengine": {origin: "www/py-qt5-webengine", version: "5.15.4_3"},
"py38-qt5-widgets": {origin: "x11-toolkits/py-qt5-widgets", version: "5.15.4_3"},
"py38-sqlite3": {origin: "databases/py-sqlite3", version: "3.8.12_7"},
"py38-xdg": {origin: "devel/py-xdg", version: "0.27"},
"py38-xmltodict": {origin: "devel/py-xmltodict", version: "0.12.0"},

We don't want py38-... in the dependencies because FreeBSD packages might switch to py39-..., at which point in time all py38-... packages should be updated to py39-... packages automatically. How to do this?

Trying to specify py3*-beautifulsoup as a dependency fails with: pkg: Missing dependency 'py3*-beautifulsoup'.

grahamperrin commented 1 year ago

… For example, we would want to depend on x11-toolkits/py-qt5-widgets but not on py38-qt5-widgets specifically, so that once py39 is there …

py39 will never be there.

grahamperrin commented 1 year ago

Re: https://github.com/helloSystem/hello/issues/161#issuecomment-973681998

https://www.freebsd.org/administration/ there's now a FreeBSD Packages Management Team.

probonopd commented 1 year ago

Here we go again. I just want to install something yet pkg wants to REMOVE something.

FreeBSD% sudo pkg install linphone
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 37 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
        FreeCAD: 0.20_1
        PrusaSlicer: 2.3.3_4
        frei0r-plugins: 1.8.0
        frei0r-plugins-opencv: 1.8.0
        gdal: 3.5.0_1
        opencv: 4.5.5_7
        openvdb: 9.1.0
        shotcut: 21.03.21_3

New packages to be INSTALLED:
        bcg729: 1.1.1 [FreeBSD]
        bcmatroska2: 0.23 [FreeBSD]
        bctoolbox: 5.1.38_1 [FreeBSD]
        bcunit: 3.0.2p20220210 [FreeBSD]
        belcard: 5.1.32 [FreeBSD]
        belle-sip: 5.1.32_1 [FreeBSD]
        belr: 5.1.32 [FreeBSD]
        bzrtp: 5.1.32 [FreeBSD]
        libantlr3c: 3.4_1 [FreeBSD]
        liblinphone: 5.1.56 [FreeBSD]
        libsrtp2: 2.4.2 [FreeBSD]
        lime: 5.0.8_1 [FreeBSD]
        linphone: 4.4.8_2,1 [FreeBSD]
        mediastreamer: 5.1.55 [FreeBSD]
        openldap26-client: 2.6.3 [FreeBSD]
        ortp: 5.1.55 [FreeBSD]
        soci: 4.0.3_2 [FreeBSD]

Installed packages to be UPGRADED:
        boost-libs: 1.79.0_1 -> 1.80.0 [FreeBSD]
        botan2: 2.19.2 -> 2.19.2_2 [FreeBSD]
        codeblocks: 20.03_2 -> 20.03_3 [FreeBSD]
        libcmis: 0.5.2_5 -> 0.5.2_6 [FreeBSD]
        libixion: 0.17.0_1 -> 0.17.0_2 [FreeBSD]
        liborcus: 0.17.2_1 -> 0.17.2_2 [FreeBSD]
        libreoffice: 7.3.5.2 -> 7.4.2.3 [FreeBSD]
        poppler: 22.06.0 -> 22.09.0 [FreeBSD]
        poppler-glib: 22.06.0 -> 22.09.0_1 [FreeBSD]
        poppler-qt5: 22.06.0 -> 22.09.0_1 [FreeBSD]
        poppler-utils: 22.06.0 -> 22.09.0_1 [FreeBSD]
        sfcgal: 1.4.1_1 -> 1.4.1_2 [FreeBSD]

Number of packages to be removed: 8
Number of packages to be installed: 17
Number of packages to be upgraded: 12

The operation will free 519 MiB.
146 MiB to be downloaded.

Proceed with this action? [y/N]:

This has to be avoided at all cost.

probonopd commented 1 year ago

Why the heck does the system want to install Qt6 out of the blue now??? In the middle of the quarter...

FreeBSD% cat /etc/pkg/FreeBSD.conf 
# $FreeBSD$
#
# To disable this repository, instead of modifying or removing this file,
# create a /usr/local/etc/pkg/repos/FreeBSD.conf file:
#
#   mkdir -p /usr/local/etc/pkg/repos
#   echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
#

FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}

FreeBSD% sudo pkg upgrade
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
Checking for upgrades (458 candidates): 100%
Processing candidates (458 candidates): 100%
The following 475 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        cmake-core: 3.24.0 [FreeBSD]
        cmake-doc: 3.24.0 [FreeBSD]
        cmake-man: 3.24.0 [FreeBSD]
        espeak-ng: 1.51.1_3 [FreeBSD]
        gdk-pixbuf-xlib: 2.40.2 [FreeBSD]
        go119: 1.19.2 [FreeBSD]
        hdf5-110: 1.10.9 [FreeBSD]
        ksanecore: 22.08.1 [FreeBSD]
        libavif: 0.10.1_5 [FreeBSD]
        libdeflate: 1.14 [FreeBSD]
        libxcvt: 0.1.2_1 [FreeBSD]
        mlt7: 7.8.0_2 [FreeBSD]
        openjdk17: 17.0.4+8.1_1 [FreeBSD]
        openldap26-client: 2.6.3 [FreeBSD]
        pcaudiolib: 1.2 [FreeBSD]
        qca-qt5: 2.3.4_1 [FreeBSD]
        qt6-5compat: 6.3.2 [FreeBSD]
        qt6-base: 6.3.2 [FreeBSD]
        qt6-svg: 6.3.2 [FreeBSD]
        qtkeychain-qt5: 0.13.2_1 [FreeBSD]
        sonivox: 3.6.11 [FreeBSD]
grahamperrin commented 1 year ago

https://github.com/helloSystem/hello/issues/161#issuecomment-1242781483

probonopd commented 1 year ago

This thread is my dumping ground for all gripes I have with pkg, and documentation why I don't think a package manager is a tool that should be given into the hands of unsuspecting non-technical end users ("mere mortals").

grahamperrin commented 1 year ago

… I don't think a package manager is a tool that should be given into the hands of unsuspecting non-technical end users …

In their hands:

probonopd commented 1 year ago

GhostBSD maintains the type of thing that helloSystem does not want to maintain:

a repository – FreeBSD ports with GhostBSD ports.

grahamperrin commented 1 year ago

dumping ground

Fair enough. Increasingly difficult to follow, GitHub issues don't lend themselves to this type of thing.

Methinks, here's no longer the place to answer questions.

grahamperrin commented 1 year ago

https://github.com/helloSystem/ISO/commit/8630218644d2a1d4f3ad1b7849b865f90f577c98 (2022-10-15) referenced https://github.com/helloSystem/hello/issues/161#issuecomment-1267976086, but not in the commit message.

probonopd commented 1 year ago

@Cozmo007 which helloSystem are you on? This should (hopefully) be fixed in the recent 0.8.0 experimental builds thanks to https://github.com/helloSystem/hello/issues/161#issuecomment-1267976086.

crees commented 1 year ago

Just a note here that it's occurred to me that Hello could mark all of the leaf packages it expects to keep as vital. This is a flag that makes pkg refuse to uninstall itself without forcing, and I think it might work to stop now missing packages from being deinstalled. I'll let you know how testing goes- going to have a go at the same time as the base package.

probonopd commented 1 year ago

Cannot build the ISO today because packages are again missing from one day to the next...

pkg-static: No packages available to install matching 'www/py-qt5-webengine' have been found in the repositories
pkg-static: No packages available to install matching 'falkon-qtonly' have been found in the repositories

Root cause possibly is (https://portsfallout.com/port/26602/, source):

=======================<phase: lib-depends    >============================
===>   py39-qt5-webengine-5.15.6_1 depends on shared library: libGL.soSegmentation fault
 - not found
===>   Installing existing package /packages/All/libglvnd-1.6.0.pkg
[124i386-quarterly-job-07] Installing libglvnd-1.6.0...
the most recent version of libglvnd-1.6.0 is already installed
===>   py39-qt5-webengine-5.15.6_1 depends on shared library: libGL.soSegmentation fault
 - not found
*** Error code 1

Similarly (source),

[124i386-quarterly-job-07] Extracting libX11-1.7.2,1: .......... done
===>   falkon-qtonly-22.12.3 depends on file: /usr/local/libdata/pkgconfig/x11.pc - found
===>   Returning to build of falkon-qtonly-22.12.3
===>   falkon-qtonly-22.12.3 depends on file: /usr/local/libdata/pkgconfig/xcb.pc - found
===========================================================================
=======================<phase: lib-depends    >============================
===>   falkon-qtonly-22.12.3 depends on shared library: libKF5Archive.so - not found
===>   Installing existing package /packages/All/kf5-karchive-5.104.0.pkg
[124i386-quarterly-job-07] Installing kf5-karchive-5.104.0...
[124i386-quarterly-job-07] Extracting kf5-karchive-5.104.0: .......... done
===>   falkon-qtonly-22.12.3 depends on shared library: libKF5Archive.soSegmentation fault
 - not found
*** Error code 1

Stop.
make: stopped in /usr/ports/www/falkon
=>> Cleaning up wrkdir
===>  Cleaning for falkon-qtonly-22.12.3
build of www/falkon@qtonly | falkon-qtonly-22.12.3 ended at Thu May 25 17:44:12 UTC 2023
build time: 00:00:13
!!! build failure encountered !!!

Any chance to get that still fixed in quarterly packages?

For me, that the previous version of each package is not kept until a new build has succeeded (like Linux distributions seem to be doing it) is the number 1 annoyance in using FreeBSD nowadays (along with the issue that graphics drivers are not treated the same as base).

grahamperrin commented 1 year ago

… packages … one day to the next … number 1 annoyance …

It seems that you want a repository, or repositories, to always fulfil the requirements of helloSystem.

From https://github.com/helloSystem/ISO/issues/141#issuecomment-1244881163 (2022-09-13):

… impossible for a single repository to completely fulfil everyone's requirements 24/7/365. …

I use poudriere to meet my requirements. I don't expect the FreeBSD Project to prioritise my requirements above those of other users (or projects).

Again, please consider using poudriere.

crees commented 1 year ago

I strongly disagree that disappearing packages are acceptable on the Project's main and recommended package repositories. You don't see Debian doing this, and portmgr AFAIK agrees that this is an undesirable state of affairs. It's simply a matter of QA, which is incredibly difficult.

grahamperrin commented 1 year ago

Any chance to get that still fixed in quarterly packages?

That's a question for the maintainer. https://www.freshports.org/www/falkon/

Postscript

Re: https://github.com/helloSystem/hello/discussions/384#discussioncomment-6237603

grahamperrin commented 1 year ago

image

This might be already amongst the thirty-seven one hundred and six hidden comments:

https://lists.freebsd.org/subscription/freebsd-pkg

https://github.com/helloSystem/hello/issues/161#issuecomment-1596242945

The suggestion to use poudriere is to be realistic about the status quo.

https://github.com/helloSystem/hello/issues/161#issuecomment-1596238688

… Root cause possibly is (portsfallout.com/port/26602, source): …

That's for 12.4-RELEASE on a different platform. https://www.freebsd.org/platforms/i386/

… previous version …

Please see:

probonopd commented 1 year ago

That's for 12.4-RELEASE on a different platform. https://www.freebsd.org/platforms/i386/

Sorry, my bad. How can I find out why it is missing for 13.1 arm64?

crees commented 1 year ago

Seems to be qt6-webengine failing.

https://pkg-status.freebsd.org/builds/default:default:131amd64:8dc4a71c010e:beefy16

I think the beefy machines that show the logs are IPv6 only, so you may have better luck than me following the links to the build logs.

probonopd commented 1 year ago

Thing is, helloSystem isn't even (trying to) use Qt 6. Hopefully we won't be forced to ship both Qt 5 and 6 in parallel... (man do I dislike major version upgrades).

Why was everything working just 2 days ago when I was able to build hello-0.8.2_0H337-FreeBSD-13.2-amd64.iso just fine? Did some packages switch from Qt 5 to Qt 5 in quarterly packages (which I expect to change only once per quarter)?

Said differently, I wish that all software that depends on Qt 5 should get a -qt6 variant that one can opt into (or not opt into, which helloSystem would like to do at this point until all packages are available with -qt6 variants).

Understandable feature request? Realistic?

grahamperrin commented 1 year ago

… quarterly packages (which I expect to change only once per quarter)? …

No, changes (merges) may occur at any time:

grahamperrin commented 1 year ago

https://github.com/helloSystem/hello/issues/161#issuecomment-1596268487

… I think the beefy machines that show the logs are IPv6 only, …

True.

https://github.com/helloSystem/hello/issues/161#issuecomment-1596262025

… How can I find out why it is missing for 13.1

Reminder:

– NB the most recent answer under 382.

arm64?

AMD64, not 64-bit ARMv8, for helloSystem at this time.

Via https://www.freebsd.org/platforms/:

grahamperrin commented 1 year ago

https://github.com/helloSystem/hello/issues/161#issuecomment-1596269465

… helloSystem isn't even (trying to) use Qt 6. …

6 was irrelevant.

Please see the two most recent answers under:

https://github.com/helloSystem/hello/discussions/384#discussioncomment-6237561 (the first of the two) identifies:

probonopd commented 1 year ago

Looks like there is at least some distant hope for the graphics drivers, as Zirias explained:

The issue is that there's always just one package repository for each major #FreeBSD version, because ABI is stable across minor releases (so, having multiple repositories is just unnecessary).

Unfortunately, this is not true for some in-kernel ABI needed by these drivers. So, the binary package can only work on one or the other minor release.

During the 3-month transition period from one to the next minor, this will be the older one for the official packages. Nevertheless, compiling the port yourself with the newer minor will work perfectly fine.

Indeed, having the drivers in base would solve this particular issue. It might happen again, main issue was licensing.

(...)

  1. It's exactly 3 months where you can't use the packages from official repositories with the newer minor release, and
  2. This can by definition never affect a .0 release, that's a new major, with a new ABI and therefore a new package repository as well.

As I said, (re-)inclusion in base will probably happen when the last piece of GPL-licensed glue-code is rewritten (the drivers themselves are unencumbered).

Until then, is it a huge problem to integrate building of a port in your scripts?

(...)

Here: https://github.com/freebsd/drm-kmod/tree/master/linuxkpi/gplv2/include

(edit: and yes, there's almost nothing left ...)

There used to be a LOT of gpl-licensed glue-code, and continuous work was done replacing it.

The drm-kmod drivers once were in base. Then, all this glue-code was required to upgrade them, so they could not live in base any more (IIRC this was around between FreeBSD 11 and 12). I now see the chance they might be reintegrated soon. But I don't work in that area myself, sorry 😉

So the way I understand it, https://github.com/freebsd/drm-kmod/tree/master/linuxkpi has two directories: "bsd" and "gplv2". All "gplv2" code needs to be converted to "bsd", so that in the end there is no more "gplv2" directory. Then the graphics drivers might have a chance of being built, tested, and released alongside base.txz (again).

The "bsd" part shouldn't be too large either because what's there is constantly reintegrated in the base kernel (and only kept for as long as the drm-kmod port must support older kernels that don't have it yet).

probonopd commented 2 months ago

Point in case why a package manager should never be in the hands of an end user, or else he might end up with a broken system:

https://youtu.be/DG7AHQhxBKY?feature=shared&t=640

He is using a helloSystem released years ago, and is now trying to use pkg (via our GUI wrappers, and via the command line), and it ends in a desaster.

We should probably remove pkg entirely from the system and only offer application bundles. Like the classic Mac, macOS, iOS, and Android do.

(This is not FreeBSD specific; a similar but worse thing happened to Linus Tech Tips: https://www.youtube.com/watch?v=0506yDSgU7M)

darkoverlordofdata commented 2 months ago

Indeed, the latest version (hello-0.9.0_0I67-FreeBSD-14.0-amd64.iso) has the same issue. I recently reinstalled it and when I install Firefox, it breaks the system.

I don’t see this issue in GhostBSD. It looks like they have their own repo, maybe they use poudriere?

On Tue, Jul 9, 2024 at 12:42 PM probonopd @.***> wrote:

Point in case why a package manager should never be in the hands of an end user, or else he might end up with a broken system:

https://youtu.be/DG7AHQhxBKY?feature=shared&t=640

He is using a helloSystem released years ago, and is now trying to use pkg (via our GUI wrappers, and via the command line), and it ends in a desaster.

We should probably remove pkg entirely from the system and only offer application bundles. Like the classic Mac, macOS, iOS, and Android do.

— Reply to this email directly, view it on GitHub https://github.com/helloSystem/hello/issues/161#issuecomment-2218517900, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJUE4S3DPLWRVYRLVNX67LZLQ4J7AVCNFSM6AAAAABKTQA4BCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJYGUYTOOJQGA . You are receiving this because you were mentioned.Message ID: @.***>

--

Bruce Davidson