New cph extracts directory permissions differently

wolfv commented 1 year ago

Checklist

[X] I added a descriptive title
[X] I searched open reports and couldn't find a duplicate

What happened?

The new conda package handling extracts directory permissions differently (sets writable bit) compared to how libarchive handled them.

The directories are actually implicitly created (on the fly). Not sure if this is serious, but might be interesting to fix (also makes the directories appear "less weird" in the terminal :)

Here is a screenshot with the new cph on top, and using the cph_test_data... package from the this repo.

Conda Info

No response

Conda Config

No response

Conda list

No response

Additional Context

No response

dholth commented 1 year ago

conda-package-handling 2.0 assumes Python tarfile would be acceptable, since conda used it until a few years ago (2018?)

This tarfile subclass helps to control permissions https://github.com/conda/conda-package-streaming/blob/main/conda_package_streaming/package_streaming.py#L32 but it should only affect chown, not chmod. Intended only to prevent https://github.com/conda/conda/blob/main/conda/gateways/disk/create.py#L217 from being necessary.

https://github.com/python/cpython/blob/3.11/Lib/tarfile.py#L2281 is the overridden method.

We also added anonymize_tarinfo to clear all uid, gid, uname, gname as 0, "" https://github.com/conda/conda-package-handling/blob/main/src/conda_package_handling/utils.py#L486 which also should not affect perms.

Do I see two shots of the new behavior in the screenshot?

dholth commented 1 year ago

On my OSX laptop, umask is 022, cph_test_data.tar.bz2 does not include explicit directory entries (that would otherwise be used to set directory permissions after tarfile.py extractall was finished), and

% cph -V
conda-package-handling 2.0.2

cph x cph_test_data-0.0.1-0.tar.bz2

% ls -lh cph_test_data-0.0.1-0
total 0
drwxr-xr-x   4 dholth  staff   128B Jan  4 17:41 bin
drwxr-xr-x  10 dholth  staff   320B Jan  4 17:41 info
drwxr-xr-x   8 dholth  staff   256B Jan  4 17:41 lib
drwxr-xr-x   3 dholth  staff    96B Jan  4 17:41 libexec
drwxr-xr-x   4 dholth  staff   128B Jan  4 17:41 share

% tar -xf ../cph_test_data-0.0.1-0.tar.bz2 
% ls -lh
drwxr-xr-x   4 dholth  staff   128B Jan  4 17:46 bin
drwxr-xr-x  10 dholth  staff   320B Jan  4 17:46 info
drwxr-xr-x   8 dholth  staff   256B Jan  4 17:46 lib
drwxr-xr-x   3 dholth  staff    96B Jan  4 17:46 libexec
drwxr-xr-x   4 dholth  staff   128B Jan  4 17:46 share

python -m tarfile -e ../cph_test_data-0.0.1-0.tar.bz2

% ls -lh t3
total 0
drwxr-xr-x   4 dholth  staff   128B Jan  4 17:45 bin
drwxr-xr-x  10 dholth  staff   320B Jan  4 17:45 info
drwxr-xr-x   8 dholth  staff   256B Jan  4 17:45 lib
drwxr-xr-x   3 dholth  staff    96B Jan  4 17:45 libexec
drwxr-xr-x   4 dholth  staff   128B Jan  4 17:45 share

wolfv commented 1 year ago

hey, sorry for little explanation on the screenshots. the first one is the new one (on Linux) that shows that the permissions are extracted differently. I tested it in a Github Codespace. The lowest one is the old conda package handling, and the center one is micromamba.

dholth commented 1 year ago

In Linux, in conda's own dockerfile container, I get permissions like

drwxr-sr-x 5 test_user test_user 4.0K Jan  5 16:32 info
drwxr-sr-x 3 test_user test_user 4.0K Jan  5 16:32 lib

dholth commented 1 year ago

I see a number of vscode remote issues related to umask? Could this be related?

dholth commented 1 year ago

Relevant libarchive flags for conda-package-handling 1.x series? https://github.com/conda/conda-package-handling/blob/1.9.0/src/conda_package_handling/archive_utils_c.c#L144

dholth commented 1 year ago

We updated conda-package-streaming (which conda-package-handling uses) to respect umask. Does https://github.com/conda/conda-package-streaming/pull/65 fix this issue?

conda / conda-package-handling