search

linux-system-roles / auto-maintenance

Linux system roles wide automated repo maintenance
MIT License
25 stars 15 forks source link

release_collection.py does not support multiple remote repos #357

Open hhenkel opened 4 days ago

hhenkel commented 4 days ago

As I'm currently trying to adapt your tooling to release our internal roles as a collection I'm facing an issue, if I have two or more repositories that have to be checked out from github. Within the release_colelction.py the lsr_role2collection.py gets executed with the --src-path argument set to the tmp folder. The issue here is, that when you have multiple remote repositories, all of the repositories are checked out to the tmp folder and the lsr_role2collection.py picks up the wrong role.

INFO:root:The role [usergroups] is updated. Getting changelog entries from [0.0.0] to [0.0.1].
DEBUG:root:/home/hhenkel/venvs/ansible-navigator/bin/python3 lsr_role2collection.py --src-owner linux-system-roles --role usergroups --src-path /tmp/collection_cu_ddjk.lsr --dest-path /tmp/ansible-collection --na
mespace my_corp --collection linux_base --subrole-prefix private_usergroups_subrole_ --readme lsr_role2collection/collection_readme.md returned 1 stdout  stderr Traceback (most recent call last):
  File "/home/hhenkel/git/github/linux-system-roles/auto-maintenance/lsr_role2collection.py", line 1824, in <module>
    sys.exit(role2collection())
             ^^^^^^^^^^^^^^^^^
  File "/home/hhenkel/git/github/linux-system-roles/auto-maintenance/lsr_role2collection.py", line 1713, in role2collection
    lsr_copytree(extra, dest)
  File "/home/hhenkel/git/github/linux-system-roles/auto-maintenance/lsr_role2collection.py", line 649, in lsr_copytree
    lsr_copytree(
  File "/home/hhenkel/git/github/linux-system-roles/auto-maintenance/lsr_role2collection.py", line 649, in lsr_copytree
    lsr_copytree(
  File "/home/hhenkel/git/github/linux-system-roles/auto-maintenance/lsr_role2collection.py", line 649, in lsr_copytree
    lsr_copytree(
  File "/home/hhenkel/git/github/linux-system-roles/auto-maintenance/lsr_role2collection.py", line 659, in lsr_copytree
    copy2(subsrc, subdest, follow_symlinks=(not symlinks))
  File "/usr/lib64/python3.12/shutil.py", line 475, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib64/python3.12/shutil.py", line 262, in copyfile
    with open(dst, 'wb') as fdst:
         ^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/tmp/ansible-collection/ansible_collections/my_corp/linux_base/sudo/.git/objects/pack/pack-05345c911920636f84a7891c8c945ab1eef70e7e.pack'

Traceback (most recent call last):
  File "/home/hhenkel/git/github/linux-system-roles/auto-maintenance/./release_collection.py", line 1089, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/hhenkel/git/github/linux-system-roles/auto-maintenance/./release_collection.py", line 1075, in main
    update_collection(args, galaxy, coll_rel)
  File "/home/hhenkel/git/github/linux-system-roles/auto-maintenance/./release_collection.py", line 662, in update_collection
    role_to_collection(
  File "/home/hhenkel/git/github/linux-system-roles/auto-maintenance/./release_collection.py", line 434, in role_to_collection
    _ = run_cmd(cmd)
        ^^^^^^^^^^^^
  File "/home/hhenkel/git/github/linux-system-roles/auto-maintenance/./release_collection.py", line 138, in run_cmd
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['/home/hhenkel/venvs/ansible-navigator/bin/python3', 'lsr_role2collection.py', '--src-owner', 'linux-system-roles', '--role', 'usergroups', '--src-path', '/tmp/collection_
cu_ddjk.lsr', '--dest-path', '/tmp/ansible-collection', '--namespace', 'my_corp', '--collection', 'linux_base', '--subrole-prefix', 'private_usergroups_subrole_', '--readme', 'lsr_role2collection/collection_read
me.md']' returned non-zero exit status 1.
richm commented 1 day ago

@hhenkel are you running this from release_collection.py, or are you running lsr_role2collection.py directly? In either case, please provide the full command line. Just a guess, but I think we need to exclude copying .git/ directory contents when processing the extras.

I don't think this is related to your problem, but --src-owner linux-system-roles doesn't seem right to me - src_owner is used in converting the roles files e.g.

...
- name: Use the role
  include_role:
    name: linux-system-roles.somerole

src_owner is used to find and replace the linux-system-roles string with the FQCN. Are you using linux-system-roles.usergroups in your code?