Error building after deleting a file: can't copy '<deleted-file> doesn't exist or not a regular file

colcon / colcon-core

Command line tool to build sets of software packages

http://colcon.readthedocs.io

Apache License 2.0

103 stars 46 forks source link

Error building after deleting a file: can't copy '<deleted-file> doesn't exist or not a regular file #633

Open tonynajjar opened 6 months ago

tonynajjar commented 6 months ago

I noticed recently that in ament_python packages, when building with --symlink-install, if I delete a file that was symlinked in the build folder, then on the subsequent build I get the error: can't copy 'build/<deleted-file> doesn't exist or not a regular file. Maybe it has to do with https://github.com/colcon/colcon-core/pull/592?

to reproduce:

colcon build --symlink-install --packages-select package
Delete file that was included in the data_files of setup.py
colcon build --symlink-install --packages-select package
Get: can't copy 'build/<deleted-file> doesn't exist or not a regular file

The workaround is simply to delete the build folder and try again but it would be nice if that wasn't necessary. Maybe you could delete automatically orphaned symlinks?

cottsay commented 4 months ago

I'm having trouble understanding exactly what the problem is here. Are you deleting the file from inside the build base or deleting the file in the package sources? Is the file still referenced in the setup.py or setup.cfg under data_files? Is data_files using a wildcard?

tonynajjar commented 4 months ago

On my phone currently so can't check in detail, but as far as I remember:

I mean deleting from the sources, e.g. from 'launch' folder
yes still referenced because the data files use a wildcard e.g. 'launch/*.launch.py'

Let me know if you can reproduce with that info or I need to provide you with a more concrete example

cottsay commented 3 months ago

Alright, I see it now.

Here's the source of the error: https://github.com/pypa/setuptools/blob/f91fa3d9fc7262e0467e4b2f84fe463f8f8d23cf/setuptools/_distutils/file_util.py#L108-L110

It's pretty much the first thing that setuptools' copy function checks, and I don't see a way around it there. I think we might be able to modify the process that populates the "staged" package and explicitly remove dangling symlinks that match a data_files wildcard like this, but it might be tricky.

Either way, there will still be a dead symlink left in the install base. Same would happen with CMake.