`--` breaks on some commands

[devidw]

ble version: 0.4.0-nightly+32f290d Bash version: 5.2.37(1)-release (aarch64-apple-darwin23.4.0)

On some commands I noticed that typing -- will break the current input and leak other output.

eg

brew list --

As soon as I've typed the -- this happens:

$ brew list -/usr/bin/awk: towc: multibyte conversion failure on: 's API for new formulae or cask data every'
-- INSERT --
 input record number 4, file
 source line number 29
--/usr/bin/awk: towc: multibyte conversion failure on: 's API for new formulae or cask data every'

 input record number 4, file
 source line number 29
/usr/bin/awk: towc: multibyte conversion failure on: 's API for new formulae or cask data every'

 input record number 4, file
 source line number 29
/usr/bin/awk: towc: multibyte conversion failure on: 's API for new formulae or cask data every'

 input record number 4, file
 source line number 29
/usr/bin/awk: towc: multibyte conversion failure on: 's API for new formulae or cask data every'

[akinomyoga]

Thanks for the report. This type of error message is specific to macOS awk, but I don't have macOS so cannot test it. MacOS awk produces error messages when it sees data that is not compatible with the current encoding. As far as I search GitHub for the data in the error message, it seems to come from the man page of brew.

• Q1: What are the results of the following commands in your environment?

$ man brew | grep 'API for new formulae' | cat -v
$ man brew | grep 'API for new formulae' | /usr/bin/awk '{print NF;}'

[devidw]

Thanks for the prompt follow-up @akinomyoga much appreciated. Also really appreciate the great work on this awesome project. ✨

As for your question, indeed I see some weird encoding in the first command output:

$ man brew | grep 'API for new formulae' | cat -v
Check HomebrewM-^@M-^Ys API for new formulae or cask data every

$ man brew | grep 'API for new formulae' | /usr/bin/awk '{print NF;}'
10

[akinomyoga]

Thanks for the result.

indeed I see some weird encoding in the first command output:

The weird encoding you see is actually what is intended by cat -v.

$ man brew | grep 'API for new formulae' | cat -v
Check HomebrewM-^@M-^Ys API for new formulae or cask data every

I actually expect M-bM-^@M-^Y if the code \u2019 is properly encoded, but the beginning byte (which is supposed to be converted to M-b by cat -v) seems to be missing in your output. It might be a characteristic of macOS cat, so I'd like to confirm the output of the following command:

• Q2: What is the result of the following command in macOS?

$ echo $'\u2019' | cat -v

$ man brew | grep 'API for new formulae' | /usr/bin/awk '{print NF;}'
10

I thought macOS /usr/bin/awk would have a problem with processing the line containing "s API for new formulae or cask data every", but the above result seems to suggest that it actually works. Maybe it is related to the locale.

• Q3: What are the results of the following commands?

$ ble/widget/display-shell-version
$ locale
$ man brew | grep 'API for new formulae' | LC_ALL=C /usr/bin/awk '{print NF;}'