bst tries to access files not available locally when using storage-service

I'm using a storage-service in the cache configuration of buildstream, and I'm hitting a few instances where buildstream tries to access the files in the cas directory without them being available.

I'm documenting here the two instances I'm currently seeing, if I ever hit something else I'll report it separately.

When staging junctions. Here is the relevant part of the backtrace:

File "/home/abderrahim/.local/pipx/venvs/buildstream/lib/python3.12/site-packages/buildstream/_pluginfactory/pluginfactory.py", line 222, in _ensure_plugin
location, defaults, display = self.get_plugin_paths(kind)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/abderrahim/.local/pipx/venvs/buildstream/lib/python3.12/site-packages/buildstream/_pluginfactory/pluginfactory.py", line 192, in get_plugin_paths
return origin.get_plugin_paths(kind, self._plugin_type)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/abderrahim/.local/pipx/venvs/buildstream/lib/python3.12/site-packages/buildstream/_pluginfactory/pluginoriginjunction.py", line 35, in get_plugin_paths
loader = self.project.loader.get_loader(self._junction, self.provenance_node)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/abderrahim/.local/pipx/venvs/buildstream/lib/python3.12/site-packages/buildstream/_loader/loader.py", line 219, in get_loader
loader = loader._get_loader(junction_name, provenance_node, load_subprojects=load_subprojects)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/abderrahim/.local/pipx/venvs/buildstream/lib/python3.12/site-packages/buildstream/_loader/loader.py", line 850, in _get_loader
element._stage_sources_at(basedir)
File "/home/abderrahim/.local/pipx/venvs/buildstream/lib/python3.12/site-packages/buildstream/element.py", line 1483, in _stage_sources_at
vdirectory._import_files_internal(import_dir, update_mtime=BST_ARBITRARY_TIMESTAMP, collect_result=False)
File "/home/abderrahim/.local/pipx/venvs/buildstream/lib/python3.12/site-packages/buildstream/storage/directory.py", line 411, in _import_files_internal
return self._import_files(
       ^^^^^^^^^^^^^^^^^^^
File "/home/abderrahim/.local/pipx/venvs/buildstream/lib/python3.12/site-packages/buildstream/storage/_filebaseddirectory.py", line 275, in _import_files
self.__import_files_from_directory(
File "/home/abderrahim/.local/pipx/venvs/buildstream/lib/python3.12/site-packages/buildstream/storage/_filebaseddirectory.py", line 427, in __import_files_from_directory
dest_subdir.__import_files_from_directory(
File "/home/abderrahim/.local/pipx/venvs/buildstream/lib/python3.12/site-packages/buildstream/storage/_filebaseddirectory.py", line 464, in __import_files_from_directory
actionfunc(src_path, dest_path, mtime, result)
File "/home/abderrahim/.local/pipx/venvs/buildstream/lib/python3.12/site-packages/buildstream/storage/_filebaseddirectory.py", line 271, in copy_action
utils.safe_copy(src_path, dest_path, result=result)
File "/home/abderrahim/.local/pipx/venvs/buildstream/lib/python3.12/site-packages/buildstream/utils.py", line 354, in safe_copy
raise UtilError("Failed to copy '{} -> {}': {}".format(src, dest, e)) from e
buildstream.utils.UtilError: Failed to copy '/home/abderrahim/.cache/buildstream/cas/objects/1d/ebc14a28db5672f73285396c832c1d6066ff9b798f8fa62972d105a141957e -> /home/abderrahim/Projects/gnome-build-meta/.bst/staged-junctions/plugins/bst-plugins-experimental.bst/085fcae8f7bcc292034a5c4d9dcc806d39c514caaff033627429de2dc341801c/requirements/plugin-requirements.txt': [Errno 2] No such file or directory: '/home/abderrahim/.cache/buildstream/cas/objects/1d/ebc14a28db5672f73285396c832c1d6066ff9b798f8fa62972d105a141957e'

When checking out to a tar. Here is the relevant part of the backtrace:

  File "/usr/lib/python3.11/site-packages/buildstream/_frontend/cli.py", line 1399, in artifact_checkout
    app.stream.checkout(
  File "/usr/lib/python3.11/site-packages/buildstream/_stream.py", line 730, in checkout
    self._export_artifact(tar, location, compression, element, hardlinks, virdir)
  File "/usr/lib/python3.11/site-packages/buildstream/_stream.py", line 772, in _export_artifact
    virdir.export_to_tar(tf, ".")
  File "/usr/lib/python3.11/site-packages/buildstream/storage/_casbaseddirectory.py", line 196, in export_to_tar
    self.open_directory(filename).export_to_tar(tarfile, arcname, mtime)
  File "/usr/lib/python3.11/site-packages/buildstream/storage/_casbaseddirectory.py", line 203, in export_to_tar
    tarinfo.size = os.path.getsize(source_name)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen genericpath>", line 50, in getsize
FileNotFoundError: [Errno 2] No such file or directory: '/root/.cache/buildstream/cas/objects/d0/ff5974b6aa52cf562bea5921840c032a860a91a3512f7fe8f768f6bbe005f6'

apache / buildstream

bst tries to access files not available locally when using storage-service #1937