search

goldstar611 / chirp

CHIRP mirror. Please open CHIRP-snap issues here!
https://github.com/goldstar611/chirp/issues
GNU General Public License v3.0
1 stars 1 forks source link

Cannot build snap - core22 etc and bypass the gnome plug incompatibilities #7

Closed abulka closed 1 year ago

abulka commented 1 year ago

Following up from the discussion https://forum.snapcraft.io/t/failed-to-release-snap/33879/7 on your workarounds to get wxpython snaps working with core22 etc and bypass the gnome plug incompatibilities. You said:

It looks like the python app you’re packing is using wxWidgets. The app I packed also uses wxWidgets, python and the gnome extension on core22. Feel free to take a look https://github.com/goldstar611/chirp/blob/master/snap/snapcraft.yaml

So, trying to use your project as a working example, which I can later copy into my project. I forked your project and tried to build it, but got an error straight away. 

snapcraft --use-lxd

Launching instance...                                                                                                                                                                 
'override-pull' in part 'chirp' failed with code 127.                                                                                                                                 
Review the scriptlet and make sure it's correct.                                                                                
Failed to execute pack in instance.                                                                                                                                                   
Full execution log: '/home/andy/.local/state/snapcraft/log/snapcraft-20230310-172025.226395.log'

the tail of the full log is 

2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.038 Extracting stage package: libxext6
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.038 Extracting stage package: libdatrie1
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.038 Extracting stage package: libglx-mesa0
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.038 Extracting stage package: libglvnd0
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.038 Extracting stage package: libwayland-cursor0
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.077 :: + sudo apt install -y curl
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.080 :: /bin/bash: line 52: sudo: command not found
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.273 'override-pull' in part 'chirp' failed with code 127.
2023-03-10 17:32:08.191 :: Review the scriptlet and make sure it's correct.
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 Traceback (most recent call last):
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318   File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/parts.py", line 171, in run
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318     aex.execute(action, stdout=stream, stderr=stream)
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/executor.py", line 304, in execute
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318     self._executor.execute(actions, stdout=stdout, stderr=stderr)
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/executor.py", line 128, in execute
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318     self._run_action(act, stdout=stdout, stderr=stderr)
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/executor.py", line 193, in _run_action
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318     handler.run_action(action, stdout=stdout, stderr=stderr)
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/part_handler.py", line 168, in run_action
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318     state = handler(step_info, stdout=stdout, stderr=stderr)
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/part_handler.py", line 194, in _run_pull
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318     self._run_step(
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/part_handler.py", line 481, in _run_step
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318     step_handler.run_scriptlet(
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/step_handler.py", line 275, in run_scriptlet
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318     raise errors.ScriptletRunError(
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 craft_parts.errors.ScriptletRunError: 'override-pull' in part 'chirp' failed with code 127.
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 Review the scriptlet and make sure it's correct.
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 The above exception was the direct cause of the following exception:
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 Traceback (most recent call last):
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318   File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 294, in _run_command
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318     _run_lifecycle_and_pack(
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318   File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 337, in _run_lifecycle_and_pack
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318     lifecycle.run(
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318   File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/parts.py", line 186, in run
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318     raise errors.PartsLifecycleError(str(err)) from err
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 snapcraft.errors.PartsLifecycleError: 'override-pull' in part 'chirp' failed with code 127.
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 Review the scriptlet and make sure it's correct.
2023-03-10 17:32:09.927 Failed to execute pack in instance.
2023-03-10 17:32:09.934 Detailed information: Run the same command again with --debug to shell into the environment if you wish to introspect this failure.
2023-03-10 17:32:09.952 Traceback (most recent call last):
2023-03-10 17:32:09.952   File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 557, in _run_in_provider
2023-03-10 17:32:09.952     instance.execute_run(cmd, check=True, cwd=output_dir)
2023-03-10 17:32:09.952   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_providers/lxd/lxd_instance.py", line 289, in execute_run
2023-03-10 17:32:09.952     return self.lxc.exec(
2023-03-10 17:32:09.952   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_providers/lxd/lxc.py", line 410, in exec
2023-03-10 17:32:09.952     return runner(final_cmd, **kwargs)  # pylint: disable=subprocess-run-check
2023-03-10 17:32:09.952   File "/snap/snapcraft/8931/usr/lib/python3.8/subprocess.py", line 516, in run
2023-03-10 17:32:09.952     raise CalledProcessError(retcode, process.args,
2023-03-10 17:32:09.952 subprocess.CalledProcessError: Command '['lxc', '--project', 'snapcraft', 'exec', 'local:snapcraft-chirp-snap-on-amd64-for-amd64-11828049', '--cwd', '/root/project', '--', 'env', 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin', 'SNAPCRAFT_MANAGED_MODE=1', 'snapcraft', 'pack', '--verbosity=brief', '--build-for', 'amd64']' returned non-zero exit status 1.
2023-03-10 17:32:09.952 Full execution log: '/home/andy/.local/state/snapcraft/log/snapcraft-20230310-172025.226395.log'
goldstar611 commented 1 year ago

/bin/bash: line 52: sudo: command not found

[Edit] sorry for the quick message. You'll need to get sudo added to the snap or just remove that line if you don't want curl.

abulka commented 1 year ago

Thanks - though I wonder how you manage to build the snap yourself, if the snap doesn't have sudo. I managed to fix it by adding

    override-pull: |
      apt update              <---- 
      apt install -y sudo     <---- 
      sudo apt install -y curl
      ....

to chirp's snapcraft.yaml.

P.S. I've used your chirp snapcraft.yaml as a template for my own project but getting an error - the build script is attempting to create a virtual environment using the venv module, but it does not have permission to write to the /root/parts/build-the-python-stuff-please/install directory.

2023-03-11 19:00:24.373 ::  to prefix=/root/parts/build-the-python-stuff-please/install/usr
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:20.300 remove directory /root/parts/build-the-python-stuff-please/build
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.125 Executing PosixPath('/root/parts/build-the-python-stuff-please/run/build.sh')
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.151 :: + python3 -m venv /root/parts/build-the-python-stuff-please/install
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.625 :: Error: [Errno 13] Permission denied: '/root/parts/build-the-python-stuff-please/install/bin/python3'
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.635 Failed to run the build script for part 'build-the-python-stuff-please'.
 build-the-python-stuff-please:
    plugin: python
    source: .

    build-environment:
      # WORKAROUND: The python plugin is broken with gnome extension
      - PATH: ${CRAFT_PART_INSTALL}/bin:${PATH}
      - PYTHONPATH: ""

    stage-packages:
      - python3-requests
      - python3-serial
      - python3-six
      - python3-suds
      - python3-future
      - python3-yattag
      - python3-wxgtk4.0
    stage:
      # WORKAROUND: Skip venv from python plugin
      - -bin/activate
      - -bin/activate.csh
      - -bin/activate.fish
      - -bin/Activate.ps1
      - -bin/python
      - -bin/python3
      - -bin/python3.10
      - -bin/pip
      - -bin/pip3
      - -bin/pip3.10
      - -pyvenv.cfg      
    python-requirements:
      - /root/project/.requirement-extras/requirements-linux-22.txt
    organize:
      usr/lib/$CRAFT_ARCH_TRIPLET/pulseaudio/libpulsecommon-15.99.so: usr/lib/$CRAFT_ARCH_TRIPLET/libpulsecommon-15.99.so
    # override-build: |
    #   install -Dm644 "snap/gui/pynsource.desktop" -t "$CRAFT_PART_INSTALL/usr/share/applications/"
    #   craftctl default

Any ideas appreciated. Perhaps this is a side effect of your workarounds removing venv (as per your comment). But then why does my project want to use venv and yours does not?

goldstar611 commented 1 year ago

CHIRP has an override-pull section but it never actually calls craftctl default (this goes hand in hand with the source: NOT_USED.zip) Is that venv error coming from the pull step?

abulka commented 1 year ago

Unlike yours, my snapcraft.yaml doesn't have an override-pull section as I want to build from source: . The venv error I think is thus coming from the build step:

2023-03-11 19:00:24.373 :: 2023-03-12 00:00:06.556 Executed: overlay gnome/sdk
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:06.556 Executing parts lifecycle: build build-the-python-stuff-please
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:06.557 Executing action
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:06.557 execute action build-the-python-stuff-please:Action(part_name='build-the-python-stuff-please', step=Step.BUILD, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:06.557 load state file: /root/parts/build-the-python-stuff-please/state/pull
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:12.486 fix artifacts: unpack_dir='/root/parts/build-the-python-stuff-please/install'
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:12.496 fix symlink: path='/root/parts/build-the-python-stuff-please/install/usr/share/X11/rgb.txt', unpack_dir='/root/parts/build-the-python-stuff-please/install', root='/root/parts/build-the-python-stuff-please/install/usr/share/X11'
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:12.526 fix symlink: target='/root/parts/build-the-python-stuff-please/install/etc/X11/rgb.txt'
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:13.056 For pkg-config file /root/parts/build-the-python-stuff-please/install/usr/share/pkgconfig/shared-mime-info.pc, prefix was changed from prefix=/usr
2023-03-11 19:00:24.373 ::  to prefix=/root/parts/build-the-python-stuff-please/install/usr
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:20.300 remove directory /root/parts/build-the-python-stuff-please/build
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.125 Executing PosixPath('/root/parts/build-the-python-stuff-please/run/build.sh')
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.151 :: + python3 -m venv /root/parts/build-the-python-stuff-please/install
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.625 :: Error: [Errno 13] Permission denied: '/root/parts/build-the-python-stuff-please/install/bin/python3'
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.635 Failed to run the build script for part 'build-the-python-stuff-please'.
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.671 Traceback (most recent call last):
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.671   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/step_handler.py", line 144, in _builtin_build
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     subprocess.run(
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/usr/lib/python3.8/subprocess.py", line 516, in run
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     raise CalledProcessError(retcode, process.args,
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 subprocess.CalledProcessError: Command '['/root/parts/build-the-python-stuff-please/run/build.sh']' returned non-zero exit status 1.
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 The above exception was the direct cause of the following exception:
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 Traceback (most recent call last):
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/parts.py", line 171, in run
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     aex.execute(action, stdout=stream, stderr=stream)
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/executor.py", line 304, in execute
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     self._executor.execute(actions, stdout=stdout, stderr=stderr)
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/executor.py", line 128, in execute
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     self._run_action(act, stdout=stdout, stderr=stderr)
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/executor.py", line 193, in _run_action
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     handler.run_action(action, stdout=stdout, stderr=stderr)
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/part_handler.py", line 168, in run_action
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     state = handler(step_info, stdout=stdout, stderr=stderr)
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/part_handler.py", line 305, in _run_build
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     self._run_step(
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/part_handler.py", line 489, in _run_step
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     return step_handler.run_builtin()
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/step_handler.py", line 106, in run_builtin
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     return handler()
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/step_handler.py", line 152, in _builtin_build
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     raise errors.PluginBuildError(part_name=self._part.name) from process_error
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 craft_parts.errors.PluginBuildError: Failed to run the build script for part 'build-the-python-stuff-please'.
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 The above exception was the direct cause of the following exception:
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 Traceback (most recent call last):
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 294, in _run_command
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     _run_lifecycle_and_pack(
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 337, in _run_lifecycle_and_pack
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     lifecycle.run(
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672   File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/parts.py", line 186, in run
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672     raise errors.PartsLifecycleError(str(err)) from err
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 snapcraft.errors.PartsLifecycleError: Failed to run the build script for part 'build-the-python-stuff-please'.
2023-03-11 19:00:25.974 Failed to execute pack in instance.
goldstar611 commented 1 year ago

Could you send me a PR to https://github.com/goldstar611/atestsnap with your snapcraft.yaml file? I'd like to see if Canonical's build servers fail in the same way. One thing I noticed was that sometimes local snap development didnt fail in the same way as the remote builders.

abulka commented 1 year ago

I've done two PR's, the first is just my full snapcraft.yaml file. The other is the entire project I'm trying to build.

Interestingly, in the full project copy, building the snap didn't give me a python3 -m venv related error but gave me a snap "pynsource": path "bin/python3" does not exist error. Not sure why the error changed - am running locally in a ubuntu 22.04 VM. There is also a github action .github/workflows/build-snap.yml which has in the past given me the same errors as locally.

goldstar611 commented 1 year ago

Hey Andy, I'm going to close the issue here but I'm curious if you ever got a sufficient workaround to some of the snap issues you faced (cant release because of symlink error, and such).

abulka commented 1 year ago

Yes I did - the new version of snapcraft 7.3.1.post21+gitfa823013 from edge or later removes the need for all workarounds. Can't wait for it to be available in stable so that GitHub Actions and other build tools can use it.

Thanks for asking!