Error when releasing a package with bloom

[TheDash]

I'm running through a bloom with this command:

bloom-release --rosdistro hydro --track hydro web_interface --edit

and it gets stuck here, outputting this and failing the bloom process:

==> git-bloom-generate -y rosrelease hydro --source upstream -i 0
Releasing packages: ['pyclearsilver', 'web_interface', 'launchman', 'ros_apache2', 'rosweb', 'image_stream', 'webui', 'web_msgs', 'rosjson']
Releasing package 'pyclearsilver' for 'hydro' to: 'release/hydro/pyclearsilver'
Releasing package 'web_interface' for 'hydro' to: 'release/hydro/web_interface'
Releasing package 'launchman' for 'hydro' to: 'release/hydro/launchman'
Releasing package 'ros_apache2' for 'hydro' to: 'release/hydro/ros_apache2'
Traceback (most recent call last):
  File "/usr/bin/git-bloom-generate", line 9, in <module>
    load_entry_point('bloom==0.5.20', 'console_scripts', 'git-bloom-generate')()
  File "/usr/lib/python2.7/dist-packages/bloom/commands/git/generate.py", line 252, in main
    run_generator(generator, args)
  File "/usr/lib/python2.7/dist-packages/bloom/commands/git/generate.py", line 164, in run_generator
    gen.post_rebase, destination)
  File "/usr/lib/python2.7/dist-packages/bloom/commands/git/generate.py", line 101, in try_execute
    retcode = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bloom/generators/release.py", line 129, in post_rebase
    return trim(trim_d)
  File "/usr/lib/python2.7/dist-packages/bloom/logging.py", line 211, in decorated
    return f(*args, **kwds)
  File "/usr/lib/python2.7/dist-packages/bloom/commands/git/patch/trim_cmd.py", line 177, in trim
    new_config = _trim(new_config, force, directory)
  File "/usr/lib/python2.7/dist-packages/bloom/commands/git/patch/trim_cmd.py", line 96, in _trim
    shutil.copytree(sub_dir, storage)
  File "/usr/lib/python2.7/shutil.py", line 208, in copytree
    raise Error, errors
Error: [('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/alias.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/alias.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/alias.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/authn_file.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/authn_file.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/authn_file.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/status.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/status.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/status.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/authz_groupfile.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/authz_groupfile.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/authz_groupfile.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/cgi.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/cgi.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/cgi.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/dir.conf', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/dir.conf', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/dir.conf'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/setenvif.conf', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/setenvif.conf', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/setenvif.conf'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/authz_user.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/authz_user.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/authz_user.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/mime.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/mime.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/mime.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/deflate.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/deflate.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/deflate.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/dir.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/dir.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/dir.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/alias.conf', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/alias.conf', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/alias.conf'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/negotiation.conf', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/negotiation.conf', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/negotiation.conf'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/env.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/env.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/env.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/auth_basic.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/auth_basic.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/auth_basic.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/negotiation.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/negotiation.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/negotiation.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/setenvif.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/setenvif.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/setenvif.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/deflate.conf', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/deflate.conf', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/deflate.conf'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/status.conf', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/status.conf', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/status.conf'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/authz_host.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/authz_host.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/authz_host.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/autoindex.conf', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/autoindex.conf', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/autoindex.conf'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/mime.conf', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/mime.conf', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/mime.conf'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/authz_default.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/authz_default.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/authz_default.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/python.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/python.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/python.load'"), ('/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/autoindex.load', '/tmp/tmpwhHvDk/ros_apache2/conf/mods-enabled/autoindex.load', "[Errno 2] No such file or directory: '/tmp/tmp8qYgjf/ros_apache2/conf/mods-enabled/autoindex.load'")]

<== Error running command '['/usr/bin/git-bloom-generate', '-y', 'rosrelease', 'hydro', '--source', 'upstream', '-i', '0']'
Release failed, exiting.

[wjwwood]

To reproduce I need what you entered for the track settings.

[TheDash]

Will do:

Upstream gbp: https://github.com/pr2-gbp/web_interface-release.git Upstream source: https://github.com/pr2/web_interface.git Branch: indigo-devel

Everything else was default.

[wjwwood]

It looks like your upstream has a bunch of symlinks and it is failing to copy those. I'll have to look into a different way to copy them so that the symlinks are preserved. I'm not sure when I'll have time to do so.

[TheDash]

I see what you mean. I'll check out the code of git-bloom-generate and see if there is anything I can do to help.

[wjwwood]

Ok, in case you have time, here are some leads to follow. The copy operation happens in two places:

• https://github.com/ros-infrastructure/bloom/blob/master/bloom/commands/git/patch/trim_cmd.py#L96
• https://github.com/ros-infrastructure/bloom/blob/master/bloom/commands/git/patch/trim_cmd.py#L113

My best guess as to what needs to be done (without first researching the error in shutil.copytree) is that you need to look over the files, find any symlinks, move them with the symlink functions in Python's os module, and add them to an exclude list before calling shutil.copytree.