5.0.6-0 rpm posttrans scriptlet failure

[phillxnet]

On a freshly created installer using 5.0.6-0 we have a rockstor %posttrans scriptlet (final install stage) failure recorded in the images zypper history:

Kiwi-ng log entrires

From a later TW build sharing the same issue re %poststrans.

Initial kiwi-ng relates scriptlets:

[ DEBUG   ]: 09:31:57 | system: (461/461) Installing: rockstor-5.0.6-0.x86_64 [.
[ DEBUG   ]: 09:31:57 | system: warning: /var/cache/kiwi/packages/Rockstor-Testing/x86_64/rockstor-5.0.6-0.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 5f043187: NOKEY
[ DEBUG   ]: 09:31:57 | system: .
[ DEBUG   ]: 09:31:57 | system: Running in chroot, ignoring command 'stop'
[ DEBUG   ]: 09:31:57 | system: No static/config-backups directory found.
[ DEBUG   ]: 09:31:57 | system: find: '/opt/rockstor/src': No such file or directory
[ DEBUG   ]: 09:31:57 | system: Running in chroot, ignoring command 'daemon-reload'
[ DEBUG   ]: 09:31:57 | system: .
[ DEBUG   ]: 09:31:57 | system: Created symlink /etc/systemd/system/multi-user.target.wants/rockstor-bootstrap.service -> /usr/lib/systemd/system/rockstor-bootstrap.service.
[ DEBUG   ]: 09:31:57 | system: done]
[ DEBUG   ]: 09:31:57 | system: Running post-transaction scripts [..
[ DEBUG   ]: 09:31:57 | system: %posttrans(coreutils-systemd-9.4-2.1.x86_64) script output:
[ DEBUG   ]: 09:31:57 | system: Header V3 RSA/SHA512 Signature, key ID 29b700a4: NOKEY
[ DEBUG   ]: 09:31:57 | system: Header SHA256 digest: OK

%posttrans scriptlet logs from kiwi-ng

[ DEBUG   ]: 09:32:03 | system: %posttrans(rockstor-5.0.6-0.x86_64) script output:
[ DEBUG   ]: 09:32:03 | system: which: no poetry in (/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin)
[ DEBUG   ]: 09:32:03 | system: creating virtual environment...
[ DEBUG   ]: 09:32:03 | system: creating shared libraries...
[ DEBUG   ]: 09:32:05 | system: upgrading shared libraries...
[ DEBUG   ]: 09:32:20 | system: installing poetry from spec 'poetry==1.7.1'...
[ DEBUG   ]: 09:32:20 | system: done! ✨ 🌟 ✨[ DEBUG   ]: 09:32:20 | system:   installed package poetry 1.7.1, installed using Python 3.11.7
[ DEBUG   ]: 09:32:20 | system:   These apps are now globally available
[ DEBUG   ]: 09:32:21 | system:     - poetry
[ DEBUG   ]: 09:32:21 | system: No config-backups-rpmsave directory found.
[ DEBUG   ]: 09:32:21 | system: Header V3 RSA/SHA512 Signature, key ID 29b700a4: NOKEY

Resulting image

2024-01-18 09:16:07 %posttrans(rockstor-5.0.6-0.x86_64) script output:
# which: no poetry in (/usr/local/bin:/home/buildbot/rockstor-buildbot/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin)
# creating virtual environment...
# creating shared libraries...
# upgrading shared libraries...
# installing poetry from spec 'poetry==1.7.1'...
# done! ✨ 🌟 ✨#   installed package poetry 1.7.1, installed using Python 3.11.5
#   These apps are now globally available
#     - poetry
# No config-backups-rpmsave directory found.
2024-01-18 09:16:33|patch  |openSUSE-2023-120|1|noarch|Leap_15_5_Updates|moderate|recommended|not-needed|applied|
2024-01-18 09:16:33|patch  |openSUSE-2024-19|1|noarch|repo-backports-update|moderate|recommended|not-needed|applied|
...
2024-01-18 09:16:33|patch  |openSUSE-SLE-15.5-2024-105|1|noarch|repo-sle-update|important|recommended|not-needed|applied|
2024-01-18 09:16:34|radd   |Leap_15_5|http://download.opensuse.org/distribution/leap/15.5/repo/oss/|
2024-01-18 09:16:34|radd   |Leap_15_5_Updates|https://download.opensuse.org/update/leap/15.5/oss/|
2024-01-18 09:16:34|radd   |repo-openh264|http://codecs.opensuse.org/openh264/openSUSE_Leap|
2024-01-18 09:16:34|radd   |home_rockstor|https://download.opensuse.org/repositories/home:/rockstor/15.5/|
2024-01-18 09:16:34|radd   |home_rockstor_branches_Base_System|https://download.opensuse.org/repositories/home:/rockstor:/branches:/Base:/System/15.5/|

This looks to be a partial execution of the build.sh script invoked from within the %posttrans scriptlet:

https://github.com/rockstor/rockstor-rpmbuild/blob/983aaef3e6cdc041c7a6433dccd0290321613c12/rockstor.spec#L436-L465

and our /opt/rockstor/poetry-install.txt, used to record our .venv creation by Poetry, looks to confirm this:

installer:~ # cat /opt/rockstor/poetry-install.txt 

[Errno 2] No such file or directory: 'python'

[phillxnet]

installer:/opt/rockstor # poetry env list
.venv (Activated)

installer:/opt/rockstor # poetry env info

Virtualenv
Python:         3.11.5
Implementation: CPython
Path:           /opt/rockstor/.venv
Executable:     /opt/rockstor/.venv/bin/python
Valid:          True

System
Platform:   linux
OS:         posix
Python:     3.11.5
Path:       /usr
Executable: /usr/bin/python3.11

installer:/opt/rockstor # ls -la /opt/rockstor/.venv/bin/python
lrwxrwxrwx 1 root root 19 Jan 18 12:28 /opt/rockstor/.venv/bin/python -> /usr/bin/python3.11

[phillxnet]

During rpmbuild in %build scriptlet we install Poetry and use it's poetry build --format sdist packaging capability. We also by default run %check which also employs this same Poetry install. But these both only concerns the build host. We do not otherwise install Poetry directly from within the rpmbuild scriptlets except via build.sh - invoked only within the %posttrans scriptlet.

So we arrive at build.sh (rockstor-core) from rockstor.spec %posttrans (after %install scriptlet has instantiated build.sh) via:

PATH="/usr/local/bin:$PATH"
export LANG=C.UTF-8
cd %{prefix}/%{name}
./build.sh

And fail in build.sh at the last line in the following section:

# Install Poetry, a dependency management, packaging, and build system.
# Uninstall legacy/transitional Poetry version of 1.1.15
if which poetry && poetry --version | grep -q "1.1.15"; then
  echo "Poetry version 1.1.15 found - UNINSTALLING"
  curl -sSL https://install.python-poetry.org | python3 - --uninstall
fi
# Install Poetry via PIPX as a global app
# https://peps.python.org/pep-0668/#guide-users-towards-virtual-environments
export PIPX_HOME=/opt/pipx  # virtual environment location, default ~/.local/pipx
export PIPX_BIN_DIR=/usr/local/bin  # binary location for pipx-installed apps, default ~/.local/bin
python3.11 -m pipx install poetry==1.7.1

# Install project dependencies defined in cwd pyproject.toml using poetry.toml
# specific configuration, i.e. virtualenv in cwd/.venv
# /opt/rockstor/.venv
# poetry env remove --all  # removes all venvs associated with a pyproject.toml
# rm -rf ~/.cache/pypoetry/virtualenvs/*  # to delete default location venvs.
# ** --no-ansi avoids special characters **
# Resolve Python 3.6 Poetry issue re char \u2022: (bullet)
# https://github.com/python-poetry/poetry/issues/3078
export LANG=C.UTF-8
export PYTHONIOENCODING=utf8
/usr/local/bin/poetry install --no-interaction --no-ansi > poetry-install.txt 2>&1

Where we are successfully pre-located, pwd wise, via the rockstor.spec cd %{prefix}/%{name} to the correct location for the actual install /opt/rockstor. But the pipx installed instance of Poetry returns:

installer:~ # cat /opt/rockstor/poetry-install.txt 

[Errno 2] No such file or directory: 'python'

[phillxnet]

From the synopsis in the last comment we can see our %posttrans altered PATH env via the which output to check for an existing Poetry, and for its version if found: in order that we might remove older version and install our preferred one.

[ DEBUG   ]: 09:32:03 | system: which: no poetry in (/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin)

We may just have insufficient activation of PATH (for pipx and consequently poetry) at this point.

i.e. on a resulting install we have:

installer:~ # which poetry
/usr/local/bin/poetry

installer:~ # ls -la /usr/local/bin/poetry
lrwxrwxrwx 1 root root 33 Jan 18 09:16 /usr/local/bin/poetry -> /opt/pipx/venvs/poetry/bin/poetry

installer:~ # ls -la /opt/pipx/venvs/poetry/bin/
total 76
drwxr-xr-x 1 root root  344 Jan 18 09:16 .
drwxr-xr-x 1 root root   92 Jan 18 09:16 ..
-rw-r--r-- 1 root root 1993 Jan 18 09:16 activate
-rw-r--r-- 1 root root  919 Jan 18 09:16 activate.csh
-rw-r--r-- 1 root root 2199 Jan 18 09:16 activate.fish
-rw-r--r-- 1 root root 9033 Jan 18 09:16 Activate.ps1
-rwxr-xr-x 1 root root  233 Jan 18 09:16 doesitcache
-rwxr-xr-x 1 root root 1217 Jan 18 09:16 dul-receive-pack
-rwxr-xr-x 1 root root 1213 Jan 18 09:16 dul-upload-pack
-rwxr-xr-x 1 root root  227 Jan 18 09:16 dulwich
-rwxr-xr-x 1 root root  227 Jan 18 09:16 keyring
-rwxr-xr-x 1 root root  250 Jan 18 09:16 normalizer
-rwxr-xr-x 1 root root  235 Jan 18 09:16 pkginfo
-rwxr-xr-x 1 root root  242 Jan 18 09:16 poetry
-rwxr-xr-x 1 root root  242 Jan 18 09:16 pyproject-build
lrwxrwxrwx 1 root root   10 Jan 18 09:16 python -> python3.11
lrwxrwxrwx 1 root root   10 Jan 18 09:16 python3 -> python3.11
lrwxrwxrwx 1 root root   19 Jan 18 09:16 python3.11 -> /usr/bin/python3.11
-rwxr-xr-x 1 root root  255 Jan 18 09:16 virtualenv

Where our poetry invocation script has the following content:

installer:~ # cat /opt/pipx/venvs/poetry/bin/poetry 
#!/opt/pipx/venvs/poetry/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from poetry.console.application import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

[Hooverdan96]

@phillxnet. with Move build.sh execution from %posttrans to rockstor-build.service #61 and Add rockstor-build systemd service #2793

will this issue be "reshaped" or even be closed because some of the cross-functional impact (posttrans & build.sh) is not valid anymore?

[phillxnet]

@Hooverdan96 Re:

will this issue be "reshaped" or even be closed because some of the cross-functional impact (posttrans & build.sh) is not valid anymore?

That was the aim of the issue you linked to actually. I found we had ended up in a tight corner with this installer failure and so set about digging us out re the re-do concerning the new rockstor-build.service. I've not re-tested yet as easiest once we have our next testing rpm. But just wanted to get that samba fix in first, then I'll have a re-test here.

But this does mean our installer needs internet access on first boot to build the .venv !! But that I think is just going to have to be the case anyway. We can add this info to the downloads page when it's next updated.

[EDIT] I'd intended to update this issue once the next testing was out and we have a confirmation either way that the changes fixed this installer issue. I think it will though.

[phillxnet]

A new installer was build using the Tumbleweed.x86_64 profile and upon installer boot, and JeOS first boot completion, we have a successful rockstor-build.service execution (build.sh):

With all services starting as expected on the subsequent reboot:

Note in the above subsequent reboot screen shot, the unmet condition for rockstor-build.service is that the .venv dir already exists: having been created during installer JeOS first boot config.

Closing as:

Fixed by rockstor-core PR: https://github.com/rockstor/rockstor-core/pull/2795 And partnered rockstor-rpmbuild PR: https://github.com/rockstor/rockstor-rpmbuild/pull/63