Default zsh startup files are inconsistant with those of bash

[lebensterben]

Description

When starting a bash shell, .bashrc would source any files under /usr/share/defaults/etc/profile, which sets various important environment variables. Whenever a new user account is created, a .bashrc would be created with those important settings.

The zsh bundle is configured with

--enable-etcdir=/usr/share/defaults/etc
--enable-zshenv=/usr/share/defaults/etc/zshenv
--enable-zlogin=/usr/share/defaults/etc/zlogin
--enable-zshrc=/usr/share/defaults/etc/zshrc

which has similar contents as those in /usr/share/defaults/etc/profile and /usr/share/defaults/etc/profile.d/*

~Although zsh is packaged, and the templates for zsh startup files, including zlogin, zshrc, and zshenv, are also available in /usr/share/defaults/etc', they're not automatically copied to$HOME` by default.~

But since these startup files are specified during configuration of the zsh bundle, and not located in zsh's default /etc directory, they're invisible to users. If a user ever set the environment variables in his user-specific zsh startup files, i.e. ~/.zshrc, ~/.zlogin or ~/.zprofile, and ~/.zshenv, he may accidentally shadowed the settings in the defaults.

Moreover, the contents of default zsh startup files are different from those in /usr/share/defaults/etc/profile.

1. Different environment variables

Compare those gcc flags in /usr/share/defaults/etc/zshenv

CFLAGS="-g -O3 -feliminate-unused-debug-types  -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wformat -Wformat-security -Wl,--copy-dt-needed-entries -m64  -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -malign-data=abi -fno-semantic-interposition -ftree-vectorize -ftree-loop-vectorize "
FFLAGS="-g -O3 -feliminate-unused-debug-types  -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -malign-data=abi -fno-semantic-interposition -ftree-vectorize  -ftree-loop-vectorize "
CFFLAGS="-g -O3 -feliminate-unused-debug-types  -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wl,--copy-dt-needed-entries -m64  -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -malign-data=abi -fno-semantic-interposition -ftree-vectorize  -ftree-loop-vectorize "
CXXFLAGS="$CFLAGS -fvisibility-inlines-hidden "

to those in /usr/share/defaults/etc/profile

CFLAGS="-g -O3 -feliminate-unused-debug-types  -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wformat -Wformat-security -m64  -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -fno-semantic-interposition -ffat-lto-objects  -fno-signed-zeros -fno-trapping-math  -fassociative-math -Wl,-sort-common -Wl,--enable-new-dtags -mtune=skylake"
FFLAGS="-g -O3 -feliminate-unused-debug-types  -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -malign-data=abi -fno-semantic-interposition -ftree-vectorize  -ftree-loop-vectorize -Wl,--enable-new-dtags "
CFFLAGS="-g -O3 -feliminate-unused-debug-types  -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -m64  -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -malign-data=abi -fno-semantic-interposition -ftree-vectorize  -ftree-loop-vectorize  -Wl,-sort-common -Wl,--enable-new-dtags "
CXXFLAGS="$CFLAGS -fvisibility-inlines-hidden -Wl,--enable-new-dtags "

On my system, the flag -Wl,--copy-dt-needed-entries has caused trouble, which would only occur when using zsh, but not bash.

These problems may result in many mysterious system behaviours, such us #670 I reported earlier, and inconsistent gcc compiling flags between bash and zsh shells, which sometimes leads to errors.

2. Certain functionalities are missing for ZSH

The files in /usr/share/defaults/etc/profile.d provide additional features to bash, but the default zsh startup files doesn't fully implement those features.

Feature	BASH	ZSH
Color	50-colors.sh would set directory colors of ls command according to user settings, if it's available; otherwise fallback to admin settings, or default settings. It also sets colors for GCC diagnostics.	zshrc colorize the output of ls, but doesn't have more granular customisation. Also it doesn't specify GCC diagnostics colors.
Command-not-found	10-command-not-found.sh notifies the user to install a certain bundle, if a command is not found on current system but available in a bundle.	Not implemented
swupd completion	50-swupd.bash provides bash completion for swupd	Not implemented
bash completion	bash_completion.sh provides bash completion for various commands	Not implemented
ccache.sh	Add /usr/lib64.ccache/bin to PATH for faster compilation	Not implemented

Suggested fix

• Make the settings in default startup files visible to users. For example, create a ~/.zshrc file with the same contents of the ~/.bashrc created for every user account, which would source /usr/share/defaults/etc/profile
• Modify /usr/share/defaults/etc/profile and `/usr/share/defaults/etc/profile.d/* to make them compatible for both zsh and bash
  • Most configurations could be directly used by zsh
  • Use the environment variables ZSH_VERSIONS and BASH_VERSION to determine the current shell, and based on that we deal with the differences between zsh and base
  • The auto-completion of bash could be utilised by zsh with

    autoload -U +X compinit && compinit
    autoload -U +X bashcompinit && bashcompinit
    source <bash_completion>

[ahkok]

the mechanism for new accounts is to fill /skel (in our case, /usr/share/defaults/etc/skel). Would that be a reasonable solution?

Honestly ideally files aren't copied, but used by default if e.g. ~/.zlogin does not exist. This would mean likely patching zsh to be that smart.

[lebensterben]

the mechanism for new accounts is to fill /skel (in our case, /usr/share/defaults/etc/skel). Would that be a reasonable solution?

Honestly ideally files aren't copied, but used by default if e.g. ~/.zlogin does not exist. This would mean likely patching zsh to be that smart.

Using /skel makes sense. An alternative way is to create a ~/.zlogin to source /etc/profile and /usr/share/defaults/etc/profile, which are the contents of ~/.bashrc. Creating this ~/.zlogin won't be a big problem because popular zsh frameworks like oh-my-zsh would not override ~/.zlogin.

To be consistent across sh, bash, zsh, and possibly fish as well, is to source /etc/profile and /usr/share/defaults/etc/profile in ~/.profile, and let each shell-specific startup file to source ~/.profile. (Currently it's ~/.profile that sources ~/.bashrc, which seems awkward).

I've tried to source those aforementioned files in zsh and I only need a few modifications for them to work:

• Use $ZSH_VERSION and $BASH_VERSION to decide whether it's zsh or bash.
• Modern zsh could work with bash completions with

  autoload -U +X compinit && compinit
  autoload -U +X bashcompinit && bashcompinit
  source <bash_completion>

• Setting prompts differently for zsh and bash

---

I'm reading those default startup files now and I already spot some problems.

• The CFLAGS in zshenv is different from the one in profile. Specifically, the flag -Wl,--copy-dt-needed-entries caused some problems for me.
• FFLAGS, CFFLAGS, andCXXFLAGS` also have this inconsistency.
• The default zsh startup files doens't add bash_completion and swupd completions to zsh.

[ahkok]

If only we had some developers to sort out this mess :)

[lebensterben]

If only we had some developers to sort out this mess :)

Well, I can make one based on /usr/share/defaults/etc/profile, and /usr/share/defaults/etc/profile.d/*.

[thiagomacieira]

stracing zsh --login, I see it does open the files correctly:

openat(AT_FDCWD, "/usr/share/defaults/etc/zshenv", O_RDONLY|O_NOCTTY) = 3</usr/share/defaults/etc/zshenv>
openat(AT_FDCWD, "/usr/share/defaults/etc/locale.conf", O_RDONLY|O_NOCTTY) = 3</usr/share/defaults/etc/locale.conf>
openat(AT_FDCWD, "/usr/lib64/zsh/5.7.1/zsh/newuser.so", O_RDONLY|O_CLOEXEC) = 3</usr/lib64/zsh/5.7.1/zsh/newuser.so>
openat(AT_FDCWD, "/home/tjmaciei/.zshenv", O_RDONLY|O_NOCTTY) = 3</home/tjmaciei/.zshenv>
openat(AT_FDCWD, "/usr/share/defaults/etc/zshenv", O_RDONLY|O_NOCTTY) = 3</usr/share/defaults/etc/zshenv>
openat(AT_FDCWD, "/usr/share/defaults/etc/locale.conf", O_RDONLY|O_NOCTTY) = 3</usr/share/defaults/etc/locale.conf>
openat(AT_FDCWD, "/usr/share/defaults/etc/zprofile", O_RDONLY|O_NOCTTY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/tjmaciei/.zprofile", O_RDONLY|O_NOCTTY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/defaults/etc/zshrc", O_RDONLY|O_NOCTTY) = 3</usr/share/defaults/etc/zshrc>

[lebensterben]

stracing zsh --login, I see it does open the files correctly: ...

@thiagomacieira thiago, I'm working on this now. And I'm not sure whether you saw our discussion on freenode.

So I plan to rewrite the default zsh startup files so that:

• zprofile would source the system wide profile, /use/share/defaults/etc/profile. I prefer to use zprofile instead of zlogin because zprofile is loaded before zshrc, and thus I can make sure the critical environment variables are set
• zhenv do nothing except sourcing /etc/zshenv to allow admin overrides. In general it's not advisible to put things here because all zsh instances will read this file
• Same for zshrc, but the reason is that the functionalities are already achieved by zprofile

In this way, I can make sure that

• Things are setting up in the same way regardless the choice of shell (among sh, bash, and zsh)
• There is no need to maintain a separate sets of zsh startup files
• Modifying the shell scripts in /use/share/defaults/etc/profile.d/* would also allow zsh to utilize the bash completion we already have

I already did some of the work but I haven't made any patch yet. Since I'm going to modify files across different repository of clear-packages, please let me know what's the best way to make those pull requests.

BTW, the current zsh bundle doesn't create a default zprofile, but a zlogin, which seems to be different on your side.

_{Sent with GitHawk}

[phmccarty]

I already did some of the work but I haven't made any patch yet. Since I'm going to modify files across different repository of clear-packages, please let me know what's the best way to make those pull requests.

You can send patches to dev@lists.clearlinux.org for the team to review.

[phmccarty]

I should also note that the contribution guidelines are located at https://github.com/clearlinux/distribution/blob/master/contributing.md

[thiagomacieira]

Any update?

[lebensterben]

Any update? ...

@thiagomacieira I will work on this now

_{Sent with GitHawk}

[lebensterben]

@thiagomacieira I sent patches to dev mailist. Please take a look when you got time.

[thiagomacieira]

I've got them marked unread so I can review and apply your changes. Thanks for the effort, BTW. I'm just trying to find the time now...

[chaoyi]

@thiagomacieira I sent patches to dev mailist. Please take a look when you got time.

Hi @lebensterben, I am a zsh user and your patch is much appreciated.

However, I think source /usr/share/defaults/etc/profile in /usr/share/defaults/etc/zshrc is not a good idea. The problem is ~/.zprofile is loaded before /usr/share/defaults/etc/zshrc. My environment variables set in ~/.zprofile with be reset to defaults in /usr/share/defaults/etc/profile.

Why not creating a /usr/share/defaults/etc/zprofile with emulate sh -c '. /usr/share/defaults/etc/profile'?

[lebensterben]

@thiagomacieira I sent patches to dev mailist. Please take a look when you got time.

Hi @lebensterben, I am a zsh user and your patch is much appreciated.

However, I think source /usr/share/defaults/etc/profile in /usr/share/defaults/etc/zshrc is not a good idea. The problem is ~/.zprofile is loaded before /usr/share/defaults/etc/zshrc. My environment variables set in ~/.zprofile with be reset to defaults in /usr/share/defaults/etc/profile.

Why not creating a /usr/share/defaults/etc/zprofile with emulate sh -c '. /usr/share/defaults/etc/profile'?

zprofile and zlogin are only sourced in login shell. If you have a graphical login manager, and the default shell is zsh, then zshrc won't be sourced because it's not interactive, but zprofile and zlogin is sourced. And I think it's the responsibility of login shells to set environment variables right.

After you logged in, when you start an instance of terminal emulator, the shell is interactive non-login shell by default. So zprofile and zlogin won't be sourced.

[chaoyi]

@thiagomacieira I sent patches to dev mailist. Please take a look when you got time.

Hi @lebensterben, I am a zsh user and your patch is much appreciated. However, I think source /usr/share/defaults/etc/profile in /usr/share/defaults/etc/zshrc is not a good idea. The problem is ~/.zprofile is loaded before /usr/share/defaults/etc/zshrc. My environment variables set in ~/.zprofile with be reset to defaults in /usr/share/defaults/etc/profile. Why not creating a /usr/share/defaults/etc/zprofile with emulate sh -c '. /usr/share/defaults/etc/profile'?

zprofile and zlogin are only sourced in login shell. If you have a graphical login manager, and the default shell is zsh, then zshrc won't be sourced because it's not interactive, but zprofile and zlogin is sourced. And I think it's the responsibility of login shells to set environment variables right.

After you logged in, when you start an instance of terminal emulator, the shell is interactive non-login shell by default. So zprofile and zlogin won't be sourced.

For interactive login shell, zprofile, .zprofile, zshrc, .zshrc are sourced in order. If source /usr/share/defaults/etc/profile is called in zshrc, system-wide default profile will be sourced after per-user profile, potentially overwriting user customization.

[lebensterben]

...

@chaoyi You need to manually start a interactive login shell with zsh -l, which is not very often to happen. Alternatively, if you login via SSH or switch to another user, it's indeed an interactive login shell. There are environment variables that only provide customisation, they'd not be critical and you can restore them with a new zsh session. There are also environment variables that set various path and flags, they might be critical. But logging into a remote session, or login locally as another user, deserves being conservative that reset environment. And also, you can restore your own settings by starting a new zsh session.

_{Sent with GitHawk}