search

canonical / craft-parts

https://canonical-craft-parts.readthedocs-hosted.com
GNU Lesser General Public License v3.0
10 stars 33 forks source link

The prime step uses the install dir when creating a list of migrateable filesets #766

Open mr-cal opened 5 days ago

mr-cal commented 5 days ago

Bug Description

craft-parts will fail if a stage and prime are defined for a part and stage removes a file.

This can be fixed by changing this line:

diff --git a/craft_parts/executor/step_handler.py b/craft_parts/executor/step_handler.py
index c88b409..29e6ef8 100644
--- a/craft_parts/executor/step_handler.py
+++ b/craft_parts/executor/step_handler.py
@@ -240,7 +240,7 @@ class StepHandler:

         else:
             files, dirs = filesets.migratable_filesets(
-                prime_fileset, str(self._part.part_install_dir)
+                prime_fileset, str(self._part.stage_dir)
             )
             files, dirs = migrate_files(
                 files=files,

I don't know if this is on purpose, but it has been like this since it was first added to craft-parts here.

source: https://github.com/canonical/snapcraft/issues/4835 from @dilyn-corner

To Reproduce

  1. Create the filetree:

    └── src
   └── foo
       └── bar
           ├── baz
           ├── buzz
           └── fizz

  2. Create and run the python script:

    
import logging
import yaml

import craft_parts from craft_parts import LifecycleManager, Step

logging.basicConfig(level=logging.DEBUG)

parts_yaml = """ parts: hello: plugin: dump source: src/ stage:

parts = yaml.safe_load(parts_yaml)

lcm = LifecycleManager( parts, application_name="example", cache_dir=".", )

with lcm.action_executor() as aex: aex.execute(lcm.plan(Step.PRIME))


### part yaml

_No response_

### Relevant log output

```shell
DEBUG:craft_parts.utils.os_utils:is_snap: False, SNAP_NAME set to None
DEBUG:craft_parts.infos:Setting target machine to x86_64
DEBUG:craft_parts.sources.local_source:ignore patterns: []
INFO:craft_parts.executor.executor:Installing build-packages
DEBUG:craft_parts.executor.executor:verify plugin environment for part 'hello'
DEBUG:craft_parts.sequencer:process hello:1
DEBUG:craft_parts.sequencer:add action hello:1(0)
DEBUG:craft_parts.sequencer:process hello:3
DEBUG:craft_parts.sequencer:add action hello:3(0)
DEBUG:craft_parts.sequencer:process hello:4
DEBUG:craft_parts.sequencer:add action hello:4(0)
DEBUG:craft_parts.sequencer:process hello:5
DEBUG:craft_parts.sequencer:add action hello:5(0)
DEBUG:craft_parts.executor.executor:execute action hello:Action(part_name='hello', step=Step.PULL, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))
DEBUG:craft_parts.executor.executor:execute action hello:Action(part_name='hello', step=Step.BUILD, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))
DEBUG:craft_parts.state_manager.states:load state file: /home/developer/dev/craft-parts-examples/prime-fileset-from-stage-dir/parts/hello/state/pull
DEBUG:craft_parts.executor.part_handler:remove directory /home/developer/dev/craft-parts-examples/prime-fileset-from-stage-dir/parts/hello/build
DEBUG:craft_parts.executor.step_handler:Executing PosixPath('/home/developer/dev/craft-parts-examples/prime-fileset-from-stage-dir/parts/hello/run/build.sh')
+ cp --archive --link --no-dereference . /home/developer/dev/craft-parts-examples/prime-fileset-from-stage-dir/parts/hello/install
DEBUG:urllib3.connectionpool:http://localhost:None "GET /v2/snaps HTTP/1.1" 200 None
DEBUG:craft_parts.executor.executor:execute action hello:Action(part_name='hello', step=Step.STAGE, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))
DEBUG:craft_parts.executor.executor:execute action hello:Action(part_name='hello', step=Step.PRIME, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))
Traceback (most recent call last):
  File "/home/developer/dev/craft-parts/craft_parts/utils/file_utils.py", line 141, in link
    os.link(source_path, destination, follow_symlinks=False)
FileNotFoundError: [Errno 2] No such file or directory: '/home/developer/dev/craft-parts-examples/prime-fileset-from-stage-dir/stage/foo/bar/baz' -> '/home/developer/dev/craft-parts-examples/prime-fileset-from-stage-dir/prime/foo/bar/baz'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/developer/dev/craft-parts-examples/prime-fileset-from-stage-dir/main.py", line 29, in <module>
    aex.execute(lcm.plan(Step.PRIME))
  File "/home/developer/dev/craft-parts/craft_parts/executor/executor.py", line 327, in execute
    self._executor.execute(actions, stdout=stdout, stderr=stderr)
  File "/home/developer/dev/craft-parts/craft_parts/executor/executor.py", line 137, in execute
    self._run_action(act, stdout=stdout, stderr=stderr)
  File "/home/developer/dev/craft-parts/craft_parts/executor/executor.py", line 214, in _run_action
    handler.run_action(action, stdout=stdout, stderr=stderr)
  File "/home/developer/dev/craft-parts/craft_parts/executor/part_handler.py", line 170, in run_action
    state = handler(step_info, stdout=stdout, stderr=stderr)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/developer/dev/craft-parts/craft_parts/executor/part_handler.py", line 407, in _run_prime
    contents = self._run_step(
               ^^^^^^^^^^^^^^^
  File "/home/developer/dev/craft-parts/craft_parts/executor/part_handler.py", line 491, in _run_step
    return step_handler.run_builtin()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/developer/dev/craft-parts/craft_parts/executor/step_handler.py", line 108, in run_builtin
    return handler()
           ^^^^^^^^^
  File "/home/developer/dev/craft-parts/craft_parts/executor/step_handler.py", line 245, in _builtin_prime
    files, dirs = migrate_files(
                  ^^^^^^^^^^^^^^
  File "/home/developer/dev/craft-parts/craft_parts/executor/migration.py", line 123, in migrate_files
    file_utils.link_or_copy(
  File "/home/developer/dev/craft-parts/craft_parts/utils/file_utils.py", line 98, in link_or_copy
    link(source, destination, follow_symlinks=follow_symlinks)
  File "/home/developer/dev/craft-parts/craft_parts/utils/file_utils.py", line 143, in link
    raise errors.CopyFileNotFound(source) from err
craft_parts.errors.CopyFileNotFound: Failed to copy '/home/developer/dev/craft-parts-examples/prime-fileset-from-stage-dir/stage/foo/bar/baz': no such file or directory.
sergiusens commented 4 days ago

the set can be created from what was staged, the file itself must come from what was staged

mr-cal commented 4 days ago

From an offline discussion with @cmatsuoka: