Use pkg_tar to flatten tar files

[gfrankliu]

I tried pkg_tar to flatten the tar files:

I created 2 test tar files:

gfrankliu@gfrankliu /tmp/test $ tar tvf file1.tar
lrwxrwxrwx gfrankliu/primarygroup 0 2024-06-21 17:30 bin -> ./usr/bin
drwxr-xr-x gfrankliu/primarygroup 0 2024-06-21 17:30 usr/
drwxr-xr-x gfrankliu/primarygroup 0 2024-06-21 17:30 usr/bin/
gfrankliu@gfrankliu /tmp/test $ tar tvf file2.tar
-rw-r--r-- gfrankliu/primarygroup 0 2024-06-21 17:11 ./bin/test

If I manually untar file1 and then file2, I get correct result:

gfrankliu@gfrankliu /tmp/test $ tar xf file1.tar
gfrankliu@gfrankliu /tmp/test $ tar xf file2.tar
gfrankliu@gfrankliu /tmp/test $ ls -l
total 28
lrwxrwxrwx 1 gfrankliu primarygroup     9 Jun 21 17:30 bin -> ./usr/bin
-rw-r--r-- 1 gfrankliu primarygroup 10240 Jun 21 17:30 file1.tar
-rw-r--r-- 1 gfrankliu primarygroup 10240 Jun 21 17:31 file2.tar
drwxr-xr-x 3 gfrankliu primarygroup  4096 Jun 21 17:30 usr
gfrankliu@gfrankliu /tmp/test $ ls -l bin/
total 0
-rw-r--r-- 1 gfrankliu primarygroup 0 Jun 21 17:11 test
gfrankliu@gfrankliu /tmp/test $ 

If I use pkg_tar:

load("@rules_pkg//:pkg.bzl", "pkg_tar")

pkg_tar(
    name = "flatten",
    deps = [
        "file1.tar",
        "file2.tar",
    ],
)  

The resulting tar tvf flatten.tar looks like this:

lrwxrwxrwx 434078/89939      0 2024-06-21 17:30 bin -> ./usr/bin
drwxr-xr-x 434078/89939      0 2024-06-21 17:30 usr/
drwxr-xr-x 434078/89939      0 2024-06-21 17:30 usr/bin/
drwxr-xr-x 434078/89939      0 2024-06-21 17:11 ./bin/
-rw-r--r-- 434078/89939      0 2024-06-21 17:11 ./bin/test

If I extract the flatten.tar (tar -xvf ...), I get

gfrankliu@gfrankliu /tmp/flatten $ ls -lR
.:
total 8
drwxr-xr-x 2 gfrankliu primarygroup 4096 Jun 21 17:11 bin
drwxr-xr-x 3 gfrankliu primarygroup 4096 Jun 21 17:30 usr

./bin:
total 0
-rw-r--r-- 1 gfrankliu primarygroup 0 Jun 21 17:11 test

./usr:
total 4
drwxr-xr-x 2 gfrankliu primarygroup 4096 Jun 21 17:30 bin

./usr/bin:
total 0

As you can see, the symlink /bin got deleted, so pkg_tar doesn't seem to create the correct flatten tar file. The exploded directories from if I manually untar file1.tar, file2.tar, vs if I untar the flatten.tar look different.

[gfrankliu]

Looks like release 1.0.0 introduced pkg_tar create_parents attribute that I can set to False to fix the issue in my test example above where "file2.tar" only includes the file, not the parent directory.

The new create_parents attribute wouldn't help if file2.tar includes the parent directory, eg, using the new file2.tar:

gfrankliu@gfrankliu /tmp/test $ tar cvf file2.tar ./bin
./bin/
./bin/test
gfrankliu@gfrankliu /tmp/test $ tar tvf file2.tar
drwxr-xr-x gfrankliu/primarygroup 0 2024-06-23 10:47 ./bin/
-rw-r--r-- gfrankliu/primarygroup 0 2024-06-23 10:47 ./bin/test

In the real world use case where

• first layer is Debian 12 that has "bin" symlinked to "usr/bin"
• second layer is a new deb package (say, iproute2) that installs some files in "bin".

The data.tar.xz from most deb files has the parent directory included. eg: data.tar.xz from iproute2 deb package has:

...
drwxr-xr-x root/root         0 2023-05-22 13:19 ./bin/
-rwxr-xr-x root/root    691016 2023-05-22 13:19 ./bin/ip
-rwxr-xr-x root/root    193680 2023-05-22 13:19 ./bin/ss
drwxr-xr-x root/root         0 2023-05-22 13:19 ./usr/bin/
-rwxr-xr-x root/root     27224 2023-05-22 13:19 ./usr/bin/lnstat
-rwxr-xr-x root/root    106952 2023-05-22 13:19 ./usr/bin/nstat
-rwxr-xr-x root/root    184936 2023-05-22 13:19 ./usr/bin/rdma
-rwxr-xr-x root/root      1658 2023-05-22 13:19 ./usr/bin/routel
...

So "bin" (parent directory) is there, just like in my new test file2.tar. Adding this new iproute2 tar would remove the Debian 12 bin -> usr/bin symlink, and create a new bin with only files from iproute2. In this case, /bin/sh will be gone, since bin->usr/bin link is gone.