Can't prime symlink in resulting oci

[gboutry]

I'm trying to build a rock with apache2 webserver. I'm hitting a problem where I'm trying to get a symlink into the final image but I can't.

I'm creating multiple symlinks in the same scriptlet, and the link using ln doesn't show up in the final image.

override-prime: |
  craftctl default
  chroot $CRAFT_PRIME
  ln -srf /usr/share/perl/5.30.0/ /usr/share/perl/5.30
  # Enable default mpm mod
  a2enmod -m -q mpm_event

a2enmod basically creates a symlink in /etc/apache2/mods-enabled. And these symlinks appear in the final image.

Inside the LXD container:

$ rockcraft -v prime --shell-after
root@rockcraft-keystone-55075201:~/project# ls -alh ../prime/etc/apache2/mods-enabled/mpm*
lrwxrwxrwx 1 root root 32 Mar  6 11:35 ../prime/etc/apache2/mods-enabled/mpm_event.conf -> ../mods-available/mpm_event.conf
lrwxrwxrwx 1 root root 32 Mar  6 11:35 ../prime/etc/apache2/mods-enabled/mpm_event.load -> ../mods-available/mpm_event.load
root@rockcraft-keystone-55075201:~/project# ls -alh ../prime/usr/share/perl/
lrwxrwxrwx  1 root root   6 Oct  5 10:27 5.30 -> 5.30.0
drwxr-xr-x 53 root root 128 Mar  6 11:56 5.30.0

After packing the rock and pushing into the docker daemon with skopeo:

$ docker run --rm -it keystone:yoga bash                                                        
root@aa3e33b7a43b:/# ls -alh /etc/apache2/mods-enabled/mpm*
lrwxrwxrwx 1 root root 32 Mar  6 11:35 /etc/apache2/mods-enabled/mpm_event.conf -> ../mods-available/mpm_event.conf
lrwxrwxrwx 1 root root 32 Mar  6 11:35 /etc/apache2/mods-enabled/mpm_event.load -> ../mods-available/mpm_event.load
root@aa3e33b7a43b:/# ls -alh /usr/share/perl/
drwxr-xr-x 53 root root 4.0K Mar  6 11:56 5.30.0

The link created with ln is missing from the docker image.

Am I missing something to get the symlink in the docker image ?

[sergiusens]

ln is not part of CRAFT_PRIME; CRAFT_PRIME is probably empty, or partially full depending on what happened in previous teps of each part, better to do ln -srf /usr/share/perl/5.30.0 $CRAFT_PRIME/usr/share/perl/5.30 and no chrooting

[gboutry]

ln is not part of CRAFT_PRIME; CRAFT_PRIME is probably empty, or partially full depending on what happened in previous teps of each part, better to do ln -srf /usr/share/perl/5.30.0 $CRAFT_PRIME/usr/share/perl/5.30 and no chooring

It's still not "primed" in the rock.

What does no chooring mean ?

[sergiusens]

sorry, I updated the comment; chroot $CRAFT_PRIME is what you pasted, if you look at the contents of prime you might see that it does not have a full filesystem. At most you might want to chroot into the overlayfs. Here's some information about how overlays work https://canonical-craft-parts.readthedocs-hosted.com/en/latest/explanation/overlays.html

[gboutry]

Ah, yes, I already removed the chroot on my side from other tries and still no link.

I also tried to use overlay-* parameters, but I consistently hit #195 when I'm on base 20.04, and I have other errors when I'm on base 22.04

[gboutry]

After using tricks from https://github.com/canonical/rockcraft/issues/195#issuecomment-1460277624, I was able to downgrade LXD to 5.9 and get overlay-packages working again.

I don't create the symlink manually anymore, it's created for me by the maintainer scripts of perl package.

  keystone:
    after: [keystone-user]
    plugin: nil
    overlay-packages:
      - apache2
      - keystone
      # Note (gboutry): required by charm-keystone-k8s / present in kolla images
      - sudo

But I still can't get that symlink inside the OCI image:

$ rockcraft prime --shell-after
root@rockcraft-keystone-52836546:~# ls -alh stage/usr/share/perl/
total 14K
drwxr-xr-x  3 root root   7 Mar  9 08:51 .
drwxr-xr-x 33 root root  55 Mar  9 08:51 ..
-rwx------  2 root root   0 Mar  9 08:50 .wh..wh..opq
-rw-r--r--  1 root root   0 Mar  9 08:51 .wh.5.30.0.dpkg-new
-rw-r--r--  1 root root   0 Mar  9 08:51 .wh.5.30.dpkg-new
lrwxrwxrwx  1 root root   6 Mar  9 08:50 5.30 -> 5.30.0
drwxr-xr-x 53 root root 253 Mar  9 08:51 5.30.0
root@rockcraft-keystone-52836546:~# ls -alh prime/usr/share/perl/
total 8.0K
drwxr-xr-x  3 root root   4 Mar  9 08:54 .
drwxr-xr-x 33 root root  33 Mar  9 08:54 ..
lrwxrwxrwx  1 root root   6 Mar  9 08:50 5.30 -> 5.30.0
drwxr-xr-x 53 root root 127 Mar  9 08:54 5.30.0

$ docker run --rm -it keystone:yoga bash                                                        
root@f4fbcf73e72d:/# ls -alh /usr/share/perl/
total 12K
drwxr-xr-x  3 root root 4.0K Mar  9 08:30 .
drwxr-xr-x  1 root root 4.0K Mar  9 08:30 ..
drwxr-xr-x 53 root root 4.0K Mar  9 08:30 5.30.0

[gboutry]

From what I understand, the problem comes from https://github.com/canonical/rockcraft/blob/57e10db63f8476431ab0ccf81751c978e026ca43/rockcraft/oci.py#L426

When it processes the path /root/prime/usr/share/perl/, the result of the first iteration is ('/root/prime/usr/share/perl', ['5.30', '5.30.0'], []), with 5.30 considered as a subdir. But since os.walk does not have followlinks set to True, it does not iterate over the symlink. Which seems sensible.

I think we should add the subdirs that are symlinks to the resulting archive. Since lots of package depend on these symlinks to properly functions.

In the case of PERL, the actual path is /usr/share/perl/5.30.0 but Perl expects /usr/share/perl/5.30, that's a trick performed in the maintainer script.

I would like to come up with a patch if you agree with this analysis.