search

ksh93 / ksh

ksh 93u+m: KornShell lives! | Latest release: https://github.com/ksh93/ksh/releases
Eclipse Public License 2.0
184 stars 31 forks source link

ksh93 performs brace expansion upon word expansions #140

Closed stephane-chazelas closed 3 years ago

stephane-chazelas commented 3 years ago 
a='{a,b}' ksh -c 'echo x{a,b} y$a z$(echo {a,b})'
ya yb za zb

(same in pdksh and derivatives).

I see that the fact that it even does it when braceexpand is off has been fixed very recently.

When discussed on the austin-group mailing list a few years ago around https://www.austingroupbugs.net/view.php?id=1193, that behaviour of doing brace expansion upon word expansions was deemed unacceptable by POSIX.

See current proposed resolution (to allow brace expansion, though not upon other expansions) in https://www.austingroupbugs.net/view.php?id=1193#c3995 (not approved yet).

McDutchie commented 3 years ago

Unfortunately, this is long-standing, historic ksh behaviour that is well documented in the manual page and in the New KornShell book. It seems likely that fixing it would break legacy scripts out there.

I don't know why POSIX are objecting now, so long after certifying ksh88 (which also behaved/behaves like this[*]) as the canonical POSIX compliant shell on multiple commercial UNIX systems. Surely they must realise that this ship sailed decades ago.

([] edit:* Correction, it seems they've removed brace expansion altogether on Solaris ksh88 a.k.a. /usr/xpg4/bin/sh. It exists on the HP-UX version of ksh88 though.)

I also don't know why this behaviour is any more unacceptable than global split and glob enabled by default for all expansions. It's all a horrible botch and we're stuck with it, unless we actively disable both split and glob in new scripts.

It's interesting that -f/-o noglob also turns off brace-expanding the result of expansions. The manual page doesn't document this as far as I can see. The book documents it as follows (page 366):

Brace Group Expansion. Any word that is subject to pathname expansion handles a comma-separated list of words enclosed in { and } specially. After field splitting, and before pathname expansion, ksh forms fields for each word with { and } by prefixing each word in the list to anything that comes before the { and appending anything that follows the }.

That suggests brace expansion should not be active at all if pathname expansion (globbing) is not active. But it remains active for non-expansion arguments. So I guess they changed that after the book was published. (I see that ksh88 behaves like the book says.)

Since brace expansion is incompatible with the current POSIX standard, the new posix option in ksh 93u+m also turns off the braceexpand option, so at least there is that.

Perhaps brace expansion itself can be fixed to act on literals only in the POSIX mode. But the standard will have to allow it to exist at all first.

stephane-chazelas commented 3 years ago

[...]

Since brace expansion is incompatible with the current POSIX standard, the new posix option in ksh 93u+m also turns off the braceexpand option, so at least there is that. [...]

Sorry, I wasn't aware of that new posix option. Then, I agree there's no issue. I'll just close that bug.

McDutchie commented 2 years ago

Perhaps brace expansion itself can be fixed to act on literals only in the POSIX mode. But the standard will have to allow it to exist at all first.

On second thought, I have now implemented that fix for posix mode, because it is also beneficial for the posix mode to improve compatibility with bash and zsh. Brace expansion is still turned off by default in posix mode, but when turned on, it will now only act on string literals.