search

canonical / snapcraft

Package, distribute, and update any app for Linux and IoT.
https://snapcraft.io
GNU General Public License v3.0
1.18k stars 440 forks source link

remote build: hangs when logging into Launchpad #4821

Closed Hook25 closed 2 weeks ago

Hook25 commented 3 months ago

Bug Description

When running snapcraft remote-build the process sometimes hangs forever on: 

remote-build is experimental and is subject to change. Use with caution.

(Example)

Re-running the build manually offline (enabling trace level logging) what seems to happen is that the last log line before the process gets stuck is 

2024-05-24 13:55:52.731 Emitter: Pausing control of the terminal

Re-trying multiple times, after this line what happens is that this line directly precedes the opening of the oauth page on launchpad. There may be some issue/race condition here?

EDIT: this seems to happen even after storing the credentials in ~/.local/share/snapcraft/launchpad-credentials EDIT2: this seems to happen sometimes in other spots, the one above is the most likely but it seems to be connected to this function somehow: (I stumbled upon this issue in a completely different step of the remote build, same function was blocking after Ctrl+C, all of the log below!)

2024-05-24 17:13:52.498   File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 1371, in do_handshake                                                                                                                       
2024-05-24 17:13:52.498     self._sslobj.do_handshake()

To Reproduce

I'm currently running

snapcraft remote-build --build-for armhf --launchpad-accept-public-upload --build-id checkbox24_build0 --verbosity trace

Without any credentials file in the launchpad-credentials, seems to fail about 50% of the times

Environment

ArchLinux, snapcraft 8.2.8 CI/CD on ubuntu/latest (alegedly, I couldn't interrupt it to get the stacktrace but it is probably the same issue)

snapcraft.yaml

https://github.com/canonical/checkbox/blob/main/checkbox-core-snap/series24/snap/snapcraft.yaml

(seems to be unrelated to the snapcraft.yaml)

Relevant log output

2024-05-24 15:07:23.834 Starting snapcraft, version 8.2.8
2024-05-24 15:07:23.834 Log verbosity level set to BRIEF
2024-05-24 15:07:23.871 Raw pre-parsed sysargs: args={'help': False, 'verbose': False, 'quiet': False, 'verbosity': 'trace', 'version': False} filtered=['remote-build', '--build-for', 'armhf', '--launchpad-accept-public-upload', '--project', 'checkbox24_build0']
2024-05-24 15:07:23.871 General parsed sysargs: command='remote-build' args=['--build-for', 'armhf', '--launchpad-accept-public-upload', '--project', 'checkbox24_build0']
2024-05-24 15:07:23.871 Configuring application...
2024-05-24 15:07:23.872 Preparing application...
2024-05-24 15:07:23.873 Command parsed sysargs: Namespace(recover=False, launchpad_accept_public_upload=True, launchpad_timeout=0, status=False, build_id=None, platform=None, build_for='armhf', project='checkbox24_build0')
2024-05-24 15:07:23.873 Build plan: platform=None, build_for=armhf
2024-05-24 15:07:23.873 Loading project file '/home/h25/prj/canonical/checkbox_clean/snap/snapcraft.yaml'
2024-05-24 15:07:23.967 CRAFT_ARCH_TRIPLET is deprecated, use CRAFT_ARCH_TRIPLET_BUILD_{ON|FOR}
2024-05-24 15:07:23.968 CRAFT_ARCH_TRIPLET is deprecated, use CRAFT_ARCH_TRIPLET_BUILD_{ON|FOR}
2024-05-24 15:07:23.968 CRAFT_ARCH_TRIPLET is deprecated, use CRAFT_ARCH_TRIPLET_BUILD_{ON|FOR}
2024-05-24 15:07:23.972 Processing grammar (on amd64 for armhf)
2024-05-24 15:07:23.972 Processing grammar for plugin: dump
2024-05-24 15:07:23.972 Processing grammar for source: .
2024-05-24 15:07:23.973 Processing grammar for override-pull: craftctl default
# version.txt created by prepare.sh
export version=`cat $CRAFT_PART_SRC/version.txt`
[ $version ] || exit 1
craftctl set version=$version
2024-05-24 15:07:23.973 Processing grammar for stage: ['version.txt']
2024-05-24 15:07:23.973 Processing grammar for plugin: nil
2024-05-24 15:07:23.973 Processing grammar for stage-packages: ['fwts']
2024-05-24 15:07:23.973 Processing grammar for after: ['version-calculator']
2024-05-24 15:07:23.973 Processing grammar for plugin: nil
2024-05-24 15:07:23.973 Processing grammar for stage-packages: ['stress-ng']
2024-05-24 15:07:23.973 Processing grammar for after: ['fwts']
2024-05-24 15:07:23.973 Processing grammar for plugin: nil
2024-05-24 15:07:23.973 Processing grammar for stage-packages: ['acpica-tools']
2024-05-24 15:07:23.973 Processing grammar for after: ['stress-ng']
2024-05-24 15:07:23.973 Processing grammar for plugin: python
2024-05-24 15:07:23.973 Processing grammar for source: checkbox-support
2024-05-24 15:07:23.973 Processing grammar for source-type: local
2024-05-24 15:07:23.973 Processing grammar for stage-packages: ['python3-bluez', 'python3-pyparsing', 'python3-requests-unixsocket', 'python3-systemd', 'libpython3-stdlib', 'libpython3.12-stdlib', 'libpython3.12-minimal', 'python3-pip', 'python3-setuptools', 'python3-wheel', 'python3-venv', 'python3-minimal', 'python3-pkg-resources', 'python3.12-minimal']
2024-05-24 15:07:23.973 Processing grammar for build-packages: ['libbluetooth-dev', 'python3-dev']
2024-05-24 15:07:23.973 Not processing grammar for non-grammar enabled keyword python-packages
2024-05-24 15:07:23.973 Processing grammar for after: ['acpi-tools']
2024-05-24 15:07:23.974 Processing grammar for stage: ['-pyvenv.cfg', '-bin/activate*', '-**/RECORD', '-**/__pycache__', '-debian']
2024-05-24 15:07:23.974 Processing grammar for build-environment: [{'C_INCLUDE_PATH': '/usr/include/python3.12'}, {'PYTHONPATH': '$CRAFT_PART_INSTALL/usr/lib/python3/dist-packages:${PYTHONPATH:-}'}, {'SETUPTOOLS_SCM_PRETEND_VERSION': '$(cat /home/h25/prj/canonical/checkbox_clean/stage/version.txt)'}]
2024-05-24 15:07:23.974 Processing grammar for override-build: craftctl default
2024-05-24 15:07:23.974 Processing grammar for plugin: python
2024-05-24 15:07:23.974 Processing grammar for source: checkbox-ng
2024-05-24 15:07:23.974 Processing grammar for source-type: local
2024-05-24 15:07:23.974 Processing grammar for build-packages: ['zlib1g-dev', 'build-essential']
2024-05-24 15:07:23.974 Processing grammar for stage-packages: ['perl', 'python3-markupsafe', 'python3-jinja2', 'python3-packaging', 'python3-padme', 'python3-requests-oauthlib', 'python3-urwid', 'python3-xlsxwriter', 'libpython3-stdlib', 'libpython3.12-stdlib', 'libpython3.12-minimal', 'python3-pip', 'python3-setuptools', 'python3-wheel', 'python3-venv', 'python3-minimal', 'python3-pkg-resources', 'python3.12-minimal']
2024-05-24 15:07:23.974 Not processing grammar for non-grammar enabled keyword python-packages
2024-05-24 15:07:23.974 Processing grammar for after: ['checkbox-support']
2024-05-24 15:07:23.974 Processing grammar for stage: ['-pyvenv.cfg', '-bin/activate*', '-**/RECORD', '-**/__pycache__', '-debian']
2024-05-24 15:07:23.974 Processing grammar for build-environment: [{'PYTHONPATH': '$CRAFT_PART_INSTALL/usr/lib/python3/dist-packages:${PYTHONPATH:-}'}, {'READTHEDOCS': 'True'}, {'SETUPTOOLS_SCM_PRETEND_VERSION': '$(cat /home/h25/prj/canonical/checkbox_clean/stage/version.txt)'}]
2024-05-24 15:07:23.974 Processing grammar for override-build: craftctl default
2024-05-24 15:07:23.974 Processing grammar for plugin: dump
2024-05-24 15:07:23.974 Processing grammar for source: providers/resource
2024-05-24 15:07:23.974 Processing grammar for source-type: local
2024-05-24 15:07:23.974 Processing grammar for stage-packages: ['cpu-checker', 'dpkg', 'dmidecode', 'libjson-xs-perl', 'pciutils', 'python3-requests-unixsocket', 'smartmontools']
2024-05-24 15:07:23.974 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.974 Processing grammar for override-build: cd src && autoreconf -i && cd -
export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-resource --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.975 Processing grammar for build-packages: ['autoconf', 'automake', 'libnl-3-dev', 'libnl-genl-3-dev', 'pkg-config']
2024-05-24 15:07:23.975 Processing grammar for after: ['checkbox-ng']
2024-05-24 15:07:23.975 Processing grammar for plugin: dump
2024-05-24 15:07:23.975 Processing grammar for source: providers/base
2024-05-24 15:07:23.975 Processing grammar for source-type: local
2024-05-24 15:07:23.975 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.975 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-base --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.975 Processing grammar for override-prime: # needed because else this will trigger the store autoreviewer and be autorejected
# cleaning this bit should not cause any issue but if it does do not only remove this line
# contact also the store/update the package because else this will always trigger
# a manual review
for libamd_path in $(find .. -name "libamdhip64.so.5*"); do
  echo "Clearing execstack on $libamd_path";
  execstack --clear-execstack "$libamd_path" || exit 1
done;
craftctl default
2024-05-24 15:07:23.975 Processing grammar for stage-packages: ['python3-opencv', 'bc', 'bonnie++', 'cryptsetup-bin', 'dbus', 'debsums', 'dmidecode', 'dmsetup', 'efibootmgr', 'ethtool', 'freeipmi-tools', 'fswebcam', 'gir1.2-cheese-3.0', 'gir1.2-clutter-1.0', 'gir1.2-gst-plugins-base-1.0', 'gir1.2-gudev-1.0', 'glmark2-es2', 'gnome-screenshot', 'gstreamer1.0-tools', 'gstreamer1.0-pulseaudio', {'on amd64': ['hdapsd']}, 'hdparm', 'i2c-tools', 'ipmitool', 'iperf', 'iperf3', 'iw', 'jq', 'kmod', 'libasound2', 'libcap2-bin', 'libfdt1', 'libsvm3', 'lsb-release', 'lshw', 'mesa-utils', 'mokutil', 'net-tools', 'nmap', 'nux-tools', 'nvme-cli', 'obexftp', 'parted', 'pciutils', 'pulseaudio-utils', 'pyotherside-doc', 'pyotherside-tests', 'qml-module-io-thp-pyotherside', 'qml6-module-io-thp-pyotherside', 'python3-dbus', 'python3-evdev', 'python3-gi', 'python3-natsort', 'python3-pil', 'python3-psutil', 'python3-pyqrcode', 'python3-serial', 'python3-yaml', 'python3-zbar', 'qml-module-qtquick-controls', 'qml-module-qtquick-layouts', 'qmlscene', 'smartmontools', 'usbutils', 'util-linux', 'uuid-runtime', 'wget', 'wmctrl', 'xz-utils', 'rt-tests', 'mdadm', {'on armhf': ['python3-rpi.gpio']}, {'on arm64': ['python3-rpi.gpio']}]
2024-05-24 15:07:23.975 Processing grammar for build-packages: ['execstack', 'libasound2-dev', 'libcap-dev']
2024-05-24 15:07:23.975 Processing grammar for organize: {'usr/lib/lib*.so*': 'usr/lib/arm-linux-gnueabihf/'}
2024-05-24 15:07:23.975 Processing grammar for after: ['checkbox-provider-resource']
2024-05-24 15:07:23.975 Processing grammar for plugin: dump
2024-05-24 15:07:23.976 Processing grammar for source: providers/docker
2024-05-24 15:07:23.976 Processing grammar for source-type: local
2024-05-24 15:07:23.976 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.976 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-docker --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.976 Processing grammar for stage-packages: ['apache2-utils']
2024-05-24 15:07:23.976 Processing grammar for after: ['checkbox-provider-base']
2024-05-24 15:07:23.976 Processing grammar for plugin: dump
2024-05-24 15:07:23.976 Processing grammar for source: providers/tpm2
2024-05-24 15:07:23.976 Processing grammar for source-type: local
2024-05-24 15:07:23.976 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.976 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-tpm2 --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.976 Processing grammar for stage-packages: ['clevis-tpm2']
2024-05-24 15:07:23.976 Processing grammar for after: ['checkbox-provider-docker']
2024-05-24 15:07:23.976 Processing grammar for plugin: dump
2024-05-24 15:07:23.976 Processing grammar for source: providers/iiotg
2024-05-24 15:07:23.976 Processing grammar for source-type: local
2024-05-24 15:07:23.976 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.977 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-iiotg --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.977 Processing grammar for after: ['checkbox-provider-tpm2']
2024-05-24 15:07:23.977 Processing grammar for plugin: dump
2024-05-24 15:07:23.977 Processing grammar for source: providers/sru
2024-05-24 15:07:23.977 Processing grammar for source-type: local
2024-05-24 15:07:23.977 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.977 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-sru --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.977 Processing grammar for after: ['checkbox-provider-iiotg']
2024-05-24 15:07:23.977 Processing grammar for plugin: dump
2024-05-24 15:07:23.977 Processing grammar for source: providers/gpgpu
2024-05-24 15:07:23.977 Processing grammar for source-type: local
2024-05-24 15:07:23.977 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.977 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-gpgpu --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.977 Processing grammar for after: ['checkbox-provider-sru']
2024-05-24 15:07:23.977 Processing grammar for plugin: dump
2024-05-24 15:07:23.977 Processing grammar for source: providers/certification-client
2024-05-24 15:07:23.977 Processing grammar for source-type: local
2024-05-24 15:07:23.977 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.977 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-certification-client --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.978 Processing grammar for after: ['checkbox-provider-gpgpu']
2024-05-24 15:07:23.978 Processing grammar for plugin: dump
2024-05-24 15:07:23.978 Processing grammar for source: providers/certification-server
2024-05-24 15:07:23.978 Processing grammar for source-type: local
2024-05-24 15:07:23.978 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.978 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-certification-server --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.978 Processing grammar for after: ['checkbox-provider-certification-client']
2024-05-24 15:07:23.978 Processing grammar for plugin: dump
2024-05-24 15:07:23.978 Processing grammar for source: providers/tutorial
2024-05-24 15:07:23.978 Processing grammar for source-type: local
2024-05-24 15:07:23.978 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.978 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-tutorial --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.978 Processing grammar for after: ['checkbox-provider-base']
2024-05-24 15:07:23.978 Processing grammar for source: https://github.com/maxwellainatchi/gnome-randr-rust.git
2024-05-24 15:07:23.978 Processing grammar for plugin: rust
2024-05-24 15:07:23.978 Not processing grammar for non-grammar enabled keyword rust-channel
2024-05-24 15:07:23.978 Processing grammar for build-packages: ['libdbus-1-dev']
2024-05-24 15:07:23.979 Processing grammar for plugin: nil
2024-05-24 15:07:23.979 Processing grammar for stage-packages: [{'on armhf': ['libraspberrypi0']}, {'on arm64': ['libraspberrypi0']}]
2024-05-24 15:07:23.979 Processing grammar for organize: {'usr/lib/lib*.so': 'usr/lib/arm-linux-gnueabihf/'}
2024-05-24 15:07:23.979 Processing grammar for plugin: nil
2024-05-24 15:07:23.979 Processing grammar for source: .
2024-05-24 15:07:23.979 Processing grammar for override-pull: craftctl default
wget https://raw.githubusercontent.com/snapcore/snapd/master/include/lk/snappy_boot_common.h
wget https://raw.githubusercontent.com/snapcore/snapd/master/include/lk/snappy_boot_v2.h
wget https://raw.githubusercontent.com/kubiko/dragonboard-gadget/20-lk/snap-boot-sel/lk-boot-env.c
2024-05-24 15:07:23.979 Processing grammar for build-packages: ['wget', {'on amd64 to armhf': ['gcc-arm-linux-gnueabihf:amd64']}, {'on amd64 to arm64': ['gcc-aarch64-linux-gnu:amd64']}]
2024-05-24 15:07:23.979 Processing grammar for override-build: mkdir -p ${CRAFT_PART_INSTALL}/bin
if [ "arm-linux-gnueabihf" = "arm-linux-gnueabihf" ]; then
    arm-linux-gnueabihf-gcc lk-boot-env.c -I/usr/include/ -Iapp/aboot -o ${CRAFT_PART_INSTALL}/bin/lk-boot-env
elif [ "arm-linux-gnueabihf" = "aarch64-linux-gnu" ]; then
    aarch64-linux-gnu-gcc lk-boot-env.c -I/usr/include/ -Iapp/aboot -o ${CRAFT_PART_INSTALL}/bin/lk-boot-env
else
    # native build
    gcc lk-boot-env.c -I/usr/include/ -Iapp/aboot -o ${CRAFT_PART_INSTALL}/bin/lk-boot-env
fi
2024-05-24 15:07:23.979 Processing grammar for plugin: nil
2024-05-24 15:07:23.979 Processing grammar for build-environment: [{'RUNTIME_VERSION': '$(cat /home/h25/prj/canonical/checkbox_clean/stage/version.txt)'}]
2024-05-24 15:07:23.979 Processing grammar for override-build: craftctl default
echo "checkbox-runtime:" >> $CRAFT_PART_INSTALL/parts_meta_info
echo "$RUNTIME_VERSION" >> $CRAFT_PART_INSTALL/parts_meta_info
2024-05-24 15:07:23.979 Processing grammar for after: ['checkbox-provider-certification-server']
2024-05-24 15:07:23.979 Processing grammar for plugin: dump
2024-05-24 15:07:23.979 Processing grammar for source: config/
2024-05-24 15:07:23.982 Running snapcraft remote-build on host
2024-05-24 15:07:23.984 Setting up RequestService
2024-05-24 15:07:23.984 Emitter: Pausing control of the terminal
2024-05-24 15:09:19.263 Emitter: Resuming control of the terminal
2024-05-24 15:09:19.263 Interrupted.
2024-05-24 15:09:19.265 Traceback (most recent call last):
2024-05-24 15:09:19.265   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/application.py", line 517, in run
2024-05-24 15:09:19.265     return_code = dispatcher.run() or 0
2024-05-24 15:09:19.265   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_cli/dispatcher.py", line 487, in run
2024-05-24 15:09:19.266     return self._loaded_command.run(self._parsed_command_args)
2024-05-24 15:09:19.266   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/commands/base.py", line 196, in run
2024-05-24 15:09:19.266     result = self._run(parsed_args, **kwargs) or result
2024-05-24 15:09:19.266   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/snapcraft/commands/remote.py", line 174, in _run
2024-05-24 15:09:19.266     self._services.remote_build.set_project(parsed_args.project)
2024-05-24 15:09:19.266   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/services/service_factory.py", line 95, in __getattr__
2024-05-24 15:09:19.266     instance.setup()
2024-05-24 15:09:19.266   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 76, in setup
2024-05-24 15:09:19.266     self.lp = self._get_lp_client()
2024-05-24 15:09:19.266   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 240, in _get_lp_client
2024-05-24 15:09:19.266     return launchpad.Launchpad.login(
2024-05-24 15:09:19.266   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/launchpad/launchpad.py", line 98, in login
2024-05-24 15:09:19.266     launchpadlib.launchpad.Launchpad.login_with(
2024-05-24 15:09:19.266   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/launchpadlib/launchpad.py", line 700, in login_with
2024-05-24 15:09:19.266     return cls._authorize_token_and_login(
2024-05-24 15:09:19.266   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/launchpadlib/launchpad.py", line 464, in _authorize_token_and_login
2024-05-24 15:09:19.267     return cls(
2024-05-24 15:09:19.267   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/launchpadlib/launchpad.py", line 230, in __init__
2024-05-24 15:09:19.267     super(Launchpad, self).__init__(
2024-05-24 15:09:19.267   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/resource.py", line 511, in __init__
2024-05-24 15:09:19.267     self._wadl = self._browser.get_wadl_application(self._root_uri)
2024-05-24 15:09:19.267   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 502, in get_wadl_application
2024-05-24 15:09:19.267     response, content = self._request(url, media_type=wadl_type)
2024-05-24 15:09:19.267   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 441, in _request
2024-05-24 15:09:19.267     response, content = self._request_and_retry(
2024-05-24 15:09:19.267   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 400, in _request_and_retry
2024-05-24 15:09:19.267     response, content = self._connection.request(
2024-05-24 15:09:19.267   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1692, in request
2024-05-24 15:09:19.267     (response, new_content) = self._request(
2024-05-24 15:09:19.267   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/launchpadlib/launchpad.py", line 144, in _request
2024-05-24 15:09:19.267     response, content = super(LaunchpadOAuthAwareHttp, self)._request(
2024-05-24 15:09:19.267   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 204, in _request
2024-05-24 15:09:19.268     return super(RestfulHttp, self)._request(
2024-05-24 15:09:19.268   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1444, in _request
2024-05-24 15:09:19.268     (response, content) = self._conn_request(conn, request_uri, method, body, headers)
2024-05-24 15:09:19.268   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1366, in _conn_request
2024-05-24 15:09:19.268     conn.connect()
2024-05-24 15:09:19.268   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1158, in connect
2024-05-24 15:09:19.268     self.sock = self._context.wrap_socket(sock, server_hostname=self.host)
2024-05-24 15:09:19.268   File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 513, in wrap_socket
2024-05-24 15:09:19.268     return self.sslsocket_class._create(
2024-05-24 15:09:19.268   File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 1100, in _create
2024-05-24 15:09:19.268     self.do_handshake()
2024-05-24 15:09:19.268   File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 1371, in do_handshake
2024-05-24 15:09:19.268     self._sslobj.do_handshake()
2024-05-24 15:09:19.268 KeyboardInterrupt
2024-05-24 15:09:19.268 Full execution log: '/home/h25/.local/state/snapcraft/log/snapcraft-20240524-150723.826893.log'

Log for EDIT2:

2024-05-24 17:06:58.985 remote: Resolving deltas: 100% (55063/55063), done. \ (413.4s)                                  ^2024-05-24 17:06:58.985 remote: Resolving deltas: 100% (55063/55063), done.                                             
2024-05-24 17:13:52.493 Interrupted.                                                                                    
2024-05-24 17:13:52.496 Traceback (most recent call last):                                                              
2024-05-24 17:13:52.496   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1396, in _conn_request                                                                                              
2024-05-24 17:13:52.496     response = conn.getresponse()                                                               
2024-05-24 17:13:52.496   File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/http/client.py", line 1375, in getresponse                                                                                                                
2024-05-24 17:13:52.496     response.begin()                                                                            
2024-05-24 17:13:52.496   File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/http/client.py", line 318, in begin                                                                                                                       
2024-05-24 17:13:52.496     version, status, reason = self._read_status()                                               
2024-05-24 17:13:52.496   File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/http/client.py", line 287, in _read_status                                                                                                                
2024-05-24 17:13:52.496     raise RemoteDisconnected("Remote end closed connection without"                             
2024-05-24 17:13:52.496 http.client.RemoteDisconnected: Remote end closed connection without response                   
2024-05-24 17:13:52.496                                                                                                 
2024-05-24 17:13:52.496 During handling of the above exception, another exception occurred:                             
2024-05-24 17:13:52.496 Traceback (most recent call last):                                                              
2024-05-24 17:13:52.496   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/application.py", line 517, in run                                                                                             
2024-05-24 17:13:52.496     return_code = dispatcher.run() or 0                                                         
2024-05-24 17:13:52.496   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_cli/dispatcher.py", line 487, in run                                                                                                      
2024-05-24 17:13:52.496     return self._loaded_command.run(self._parsed_command_args)                                  
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/commands/base.py", line 196, in run                                                                                           
2024-05-24 17:13:52.497     result = self._run(parsed_args, **kwargs) or result                                         
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/snapcraft/commands/remote.py", line 235, in _run                                                                                                
2024-05-24 17:13:52.497     builds = builder.start_builds(                                                              
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 130, in start_builds                                                                           
2024-05-24 17:13:52.497     self._recipe = self._ensure_recipe(                                                         
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 303, in _ensure_recipe                                                                         
2024-05-24 17:13:52.497     recipe = self._get_recipe()                                                                 
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 332, in _get_recipe                                                                            
2024-05-24 17:13:52.497     return self.RecipeClass.get(                                                                
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/launchpad/models/recipe.py", line 288, in get                                                                                 
2024-05-24 17:13:52.497     lp.lp.snaps.getByName(owner=util.get_person_link(owner), name=name),                        
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/resource.py", line 642, in __call__                                                                                          
2024-05-24 17:13:52.497     response, content = self.root._browser._request(                                            
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 441, in _request                                                                                          
2024-05-24 17:13:52.497     response, content = self._request_and_retry(                                                
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 400, in _request_and_retry                                                                                
2024-05-24 17:13:52.497     response, content = self._connection.request(                                               
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1724, in request                                                                                                    
2024-05-24 17:13:52.497     (response, content) = self._request(                                                        
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/launchpadlib/launchpad.py", line 144, in _request                                                                                               
2024-05-24 17:13:52.497     response, content = super(LaunchpadOAuthAwareHttp, self)._request(                          
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 204, in _request                                                                                          
2024-05-24 17:13:52.497     return super(RestfulHttp, self)._request(                                                   
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1444, in _request                                                                                                   
2024-05-24 17:13:52.497     (response, content) = self._conn_request(conn, request_uri, method, body, headers)          
2024-05-24 17:13:52.497   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1405, in _conn_request                                                                                              
2024-05-24 17:13:52.497     conn.connect()                                                                              
2024-05-24 17:13:52.498   File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1158, in connect                                                                                                    
2024-05-24 17:13:52.498     self.sock = self._context.wrap_socket(sock, server_hostname=self.host)                      
2024-05-24 17:13:52.498   File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 513, in wrap_socket
2024-05-24 17:13:52.498     return self.sslsocket_class._create(                                                        
2024-05-24 17:13:52.498   File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 1100, in _create   
2024-05-24 17:13:52.498     self.do_handshake()                                                                         
2024-05-24 17:13:52.498   File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 1371, in do_handshake                                                                                                                       
2024-05-24 17:13:52.498     self._sslobj.do_handshake()

Additional context

No response

mr-cal commented 3 weeks ago

This should be resolved as part of https://github.com/canonical/craft-application/issues/337

mr-cal commented 2 weeks ago

We've made some improvements to retrying API calls to Launchpad that may help for the 'EDIT2' report. I haven't been able to reproduce this locally.

Concerning the hanging when logging in in a CI instance, unfortunately we're not going to address that. We've decided not to support using remote-build in CI. CI builds for snaps should be orchestrated via snapcraft.io.