search

akinomyoga / ble.sh

Bash Line Editor―a line editor written in pure Bash with syntax highlighting, auto suggestions, vim modes, etc. for Bash interactive sessions.
BSD 3-Clause "New" or "Revised" License
2.33k stars 77 forks source link

Errors and failure from `bleopt complete_menu_style` #458

Open pallaswept opened 3 weeks ago

pallaswept commented 3 weeks ago

ble version: 0.4.0-nightly+365101c Bash version: 5.2.26(1)-release (x86_64-suse-linux)

Trying to set this option, an error is generated, that the function is undefined. But, it is definitely defined in ble.sh, and can be manually executed.

Either ble-reload, or manually entering the bleopt command exactly as it exists in init.sh, the setting is now accepted without error, however the completion descriptions appear broken:

bleopt: Invalid value complete_menu_style='desc-text'.
  A function 'ble/complete/menu-style:desc-text/construct-page' is not defined.

[ 11:31:10.342 09/06 ] /home/pallaswept
> bleopt complete_menu_style
bleopt complete_menu_style=align-nowrap
[ 11:31:10.373 + 0ms @ 100.0% CPU ]  bleopt complete_menu_style

[ 11:31:41.933 09/06 ] /home/pallaswept
> rg '^bleopt complete_menu' /home/pallaswept/.config/blesh/init.sh
43:bleopt complete_menu_style=desc-text
[ 11:31:41.955 + 3ms @ 114.2% CPU ]  rg '^bleopt complete_menu' /home/pallaswept/.config/blesh/init.sh

[ 11:32:34.976 09/06 ] /home/pallaswept
> bleopt complete_menu_style=desc-text
[ 11:32:34.997 + 0ms @ 100.0% CPU ]  bleopt complete_menu_style=desc-text

~                                                                 was 20ms                                                                                                                                                                                                 
> rg -.
-.                        : ^[[0;1m(action:progcomp)^[[m          | --hidden                       : ^[[0;1m(action:progcomp)^[[m     | --no-filename                     : ^[[0;1m(action:progcomp)^[[m  | --pcre2-unicode    : ^[[0;1m(action:progcomp)^[[m
-0                        : ^[[0;1m(action:progcomp)^[[m          | --hostname-bin                 : ^[[0;1m(action:progcomp)^[[m     | --no-fixed-strings                : ^[[0;1m(action:progcomp)^[[m  | --pcre2-version    : ^[[0;1m(action:progcomp)^[[m
-a                        : ^[[0;1m(action:progcomp)^[[m          | --hyperlink-format             : ^[[0;1m(action:progcomp)^[[m     | --no-follow                       : ^[[0;1m(action:progcomp)^[[m  | --pre              : ^[[0;1m(action:progcomp)^[[m
-A                        : ^[[0;1m(action:progcomp)^[[m          | -i                             : ^[[0;1m(action:progcomp)^[[m     | --no-glob-case-insensitive        : ^[[0;1m(action:progcomp)^[[m  | --pre-glob         : ^[[0;1m(action:progcomp)^[[m
--after-context           : ^[[0;1m(action:progcomp)^[[m          | -I                             : ^[[0;1m(action:progcomp)^[[m     | --no-heading                      : ^[[0;1m(action:progcomp)^[[m  | --pretty           : ^[[0;1m(action:progcomp)^[[m

image

I guess this is some kind of timing issue? Thanks in advance for any help! :)

akinomyoga commented 3 weeks ago

Can you use desc instead of desc-text? The menu style desc-text is not intended to be used with the menu for the default completions.

Trying to set this option, an error is generated, that the function is undefined. But, it is definitely defined in ble.sh, and can be manually executed.

To specify the menu-style desc-text, you need to do it after the module defining the menu-style desc-text (i.e. core-complete) is loaded. To set it up in ~/.blerc, you need to follow the description at the beginning of §7.

You can specify the other menu styles in ~/.blerc directly because auto-loader functions are defined for them.

however the completion descriptions appear broken:

Use desc. This is the intentional behavior of desc-text. The menu-style desc-text is used to show all the characters including the terminal escape sequences in a visible form. However, the default completion generates the descriptions with colors and styles using the terminal escape sequences. The descriptions generated by the default completion assumes desc. If you want to use desc-text, you need to override all the completion generation with your custom one that it doesn't include any terminal escape sequences.

pallaswept commented 3 weeks ago

Can you use desc instead of desc-text? The menu style desc-text is not intended to be used with the menu for the default completions.

Sorry I didn't mention that I had tried this. I do not see the escape characters like this, but the descriptions are still missing:

> rg -.
-.                        : (action:progcomp)                     | --hidden                       : (action:progcomp)                | --no-filename                     : (action:progcomp)             | --pcre2-unicode    : (action:progcomp)
-0                        : (action:progcomp)                     | --hostname-bin                 : (action:progcomp)                | --no-fixed-strings                : (action:progcomp)             | --pcre2-version    : (action:progcomp)
-a                        : (action:progcomp)                     | --hyperlink-format             : (action:progcomp)                | --no-follow                       : (action:progcomp)             | --pre              : (action:progcomp)

Thanks for explaining the difference between the two options. I will use desc from now on. I misunderstood the docs, when it mentioned leaving the escape characters, I thought that implied all the colour from ble.sh. I understand now, I think :laughing:

Thanks also for the reminder that I should have been using delayed loading. I tried taking that approach, and now I don't get the 'invalid value' or 'not defined' errors mentioned in the first post, but I still have the missing descriptions shown here. I assumed that I am setting it up incorrectly, but I'm not sure, because it seems to work sometimes, but not others. For example:

If I type fzf (f,z,f,space) and hit tab, I see this (note the description is present for the +i option at the end)

> fzf .Trash-0
.Trash-0   : directory                                            | etc   : directory                                                 | proc : directory                                                  | tmp : directory (sticky)
.snapshots : directory                                            | home  : directory                                                 | root : directory                                                  | usr : directory
bin        : symbolic link                                        | lib   : symbolic link                                             | run  : directory                                                  | var : directory
boot       : directory                                            | lib64 : symbolic link                                             | sbin : symbolic link                                              | +i  : Case‐sensitive match
cmd-output : file                                                 | mnt   : directory                                                 | srv  : directory                                                  | +m  : Disable multi‐select
dev        : directory                                            | opt   : directory                                                 | sys  : directory                                                  | +s  : Do not sort the result

If I type fzf - (add hyphen for context) and hit tab, the description for -i is as above. (these aren't the same 'i' option, to be clear)

> fzf --
--                 : (action:progcomp)                            | --filter       : (action:progcomp)                                | --min-height        : (action:progcomp)                           | --print0      : (action:progcomp)
-0                 : (action:progcomp)                            | -h             : (action:progcomp)                                | --multi             : (action:progcomp)                           | --print-query : (action:progcomp)
-1                 : (action:progcomp)                            | --header       : (action:progcomp)                                | -n                  : (action:progcomp)                           | --prompt      : (action:progcomp)
--ansi             : (action:progcomp)                            | --header-first : (action:progcomp)                                | --no-bold           : (action:progcomp)                           | -q            : (action:progcomp)
--bind             : (action:progcomp)                            | --header-lines : (action:progcomp)                                | --no-clear          : (action:progcomp)                           | --query       : (action:progcomp)
--border           : (action:progcomp)                            | --height       : (action:progcomp)                                | --no-color          : (action:progcomp)                           | --read0       : (action:progcomp)
--border-label     : (action:progcomp)                            | --help         : (action:progcomp)                                | --no-extended       : (action:progcomp)                           | --reverse     : (action:progcomp)
--border-label-pos : (action:progcomp)                            | --history      : (action:progcomp)                                | --no-hscroll        : (action:progcomp)                           | --scheme      : (action:progcomp)
--color            : (action:progcomp)                            | --history-size : (action:progcomp)                                | --no-mouse          : (action:progcomp)                           | --scroll-off  : (action:progcomp)
--cycle            : (action:progcomp)                            | --hscroll-off  : (action:progcomp)                                | --no-scrollbar      : (action:progcomp)                           | --select-1    : (action:progcomp)
-d                 : (action:progcomp)                            | -i             : (action:progcomp)                                | --no-separator      : (action:progcomp)                           | --separator   : (action:progcomp)
--delimiter        : (action:progcomp)                            | --info         : (action:progcomp)                                | --no-sort           : (action:progcomp)                           | --sync        : (action:progcomp)
--disabled         : (action:progcomp)                            | --jump-labels  : (action:progcomp)                                | --no-unicode        : (action:progcomp)                           | --tabstop     : (action:progcomp)
-e                 : (action:progcomp)                            | --keep-right   : (action:progcomp)                                | --nth               : (action:progcomp)                           | --tac         : (action:progcomp)
--ellipsis         : (action:progcomp)                            | --layout       : (action:progcomp)                                | --padding           : (action:progcomp)                           | --tiebreak    : (action:progcomp)
--exact            : (action:progcomp)                            | --listen       : (action:progcomp)                                | --pointer           : (action:progcomp)                           | --track       : (action:progcomp)
--exit-0           : (action:progcomp)                            | --literal      : (action:progcomp)                                | --preview           : (action:progcomp)                           | --version     : (action:progcomp)
--expect           : (action:progcomp)                            | -m             : (action:progcomp)                                | --preview-label     : (action:progcomp)                           | --with-nth    : (action:progcomp)
-f                 : (action:progcomp)                            | --margin       : (action:progcomp)                                | --preview-label-pos : (action:progcomp)                           | 
--filepath-word    : (action:progcomp)                            | --marker       : (action:progcomp)                                | --preview-window    : (action:progcomp)                           |

I figured it must be 'pilot error' but when I saw this inconsistency, I had second thoughts. Again, my apologies if this is a bad setup on my behalf.

akinomyoga commented 3 weeks ago

Sorry I didn't mention that I had tried this. I do not see the escape characters like this, but the descriptions are still missing:

Ah, OK. That happens when ble.sh fails to extract the descriptions from the man page. The string (action:progcomp) is the string shown when no corresponding description is found for the option, and it tells that the option is suggested by the programmable completion (progcomp).

$ ls -lt "$_ble_base_cache/complete.mandb/$LANG" | head

One possibility is that the man page of rg and fzf are not prepared in your system.

Another possibility is that the format of the man pages is totally different in your system.

pallaswept commented 3 weeks ago

I think you are on to the problem immediately. The manpages are installed and seem to be working, but the cache seems incomplete:

> \ls -lt "$_ble_base_cache/complete.mandb/$LANG"
total 32
-rw-r--r-- 1 pallaswept pallaswept    87 Jun  9 13:37 fzf
drwxr-xr-x 1 pallaswept pallaswept    28 Jun  9 13:37 man.d
-rw-r--r-- 1 pallaswept pallaswept 21523 Jun  9 13:35 systemctl
-rw-r--r-- 1 pallaswept pallaswept   561 Jun  9 13:34 cd
drwxr-xr-x 1 pallaswept pallaswept     4 Jun  9 13:34 help.d

> bat "$_ble_base_cache/complete.mandb/$LANG"/fzf
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: /home/pallaswept/.cache/blesh/0.4/complete.mandb/en_GB.UTF-8/fzf
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ +i^\^\ ^\Case‐sensitive match
   2   │ +m^\^\ ^\Disable multi‐select
   3   │ +s^\^\ ^\Do not sort the result
───────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

I notice rg is absent, I did try to trigger completion for it, but still it remains the same. I tried systemctl and cd earlier when I was posting to you. I checked that I have write perms there and I do. So, I refreshed the manpage databse and tried to trigger autocompletion for bat, which ... half-worked? I am unsure what this means.

> sudo mandb
Purging old database entries in /usr/share/man...
...
0 man subdirectories contained newer manual pages.
0 manual pages were added.
0 stray cats were added.
0 old database entries were purged.

> bat --color
--color <when>        : Specify when to use colored output. The … | --language <language>                      [-l] highlights lines… | --show-all                 : [-A] Show non-printable characters … | --config-dir           : (action:progcomp)
--decorations <when>  : Specify when to use the decorations that… | --line-range :40                           prints lines 1 to 40   | --style <style-components> : Configure which elements (line numb… | --config-file          : (action:progcomp)
--diff                : [-d] Only show lines that have been adde… | --list-languages                           [-L] Display a list o… | --tabs <T>                 : Set the tab width to T spaces. Use … | --diagnostic           : (action:progcomp)
--diff-context <N>... : Include N lines of context around added/… | --list-themes                              Display a list of sup… | --terminal-width <width>   : Explicitly set the width of the ter… | --generate-config-file : (action:progcomp)
--file-name <name>... : Specify the name to display for a file. … | --map-syntax <glob‐pattern:syntax‐name>... [-m] Map a glob patte… | --theme <theme>            : Set the theme for syntax highlighti… | --ignored-suffix       : (action:progcomp)
--force-colorization  : [-f] Alias for ’‐‐decorations=always ‐‐c… | --number                                   [-n] Only show line n… | --version                  : [-V] Show version information.       | --lessopen             : (action:progcomp)
--help                : [-h] Print this help message. -V, --vers… | --pager <command>                          Determine which pager… | --wrap <mode>              : Specify the text-wrapping mode (*au… | 
--highlight-line 40   : highlights line 40                        | --paging <when>                            Specify when to use t… | --acknowledgements         : (action:progcomp)                    | 
--italic-text <when>  : Specify when to use ANSI sequences for i… | --plain                                    [-p] Only show plain … | --cache-dir                : (action:progcomp)                    |
akinomyoga commented 3 weeks ago

I first need to clarify that there is no standard format for the man page for defining the options and their descriptions, so one would need to use heuristic matching to the options and their descriptions (to the text or the source of the man page). ble.sh uses a rather conservative approach so that it doesn't pick wrong strings that just look like options and their description.

I confirmed that the options cannot be extracted from the man page of rg in my environment, Fedora 39. This means that the man page of rg is not formatted in a way known to ble.sh. I can later analyze the formatting used by the rg man page and add support for it to ble.sh.

For the fzf completion, ble.sh can extract the descriptions in my environment. The descriptions of all the options starting with -- seem to be extracted from the man page. One possibility is that the format of the man page was changed at a certain version of fzf. The version of the fzf package in my environment, Fedora 39, is 0.44.1 (Fedora).

Or another possibility is that the distribution (either mine or yours) modifies the man page, so the format for options and descriptions is different.

for bat, which ... half-worked? I am unsure what this means.

I confirmed the behavior in my environment, but this seems to be just because the upstream man page provided by bat is outdated. See this. It doesn't contain the descriptions of those options (--acknoledgements, --cache-dir, etc.).

pallaswept commented 3 weeks ago

OK, it seems like we've fixed everything here, and any further bugs are likely all upstream, so thank you for your help! I'm glad to know that my blesh config appears to be behaving as expected now.

Looking at fzf, my version is 0.52 (0.52.0), and the build for opensuse does not modify the man pages. I think that around this time is when fzf added native tmux support, so I guess maybe the man page did change?

Anyway, I do understand that it won't be possible to parse every option from every app's manpages perfectly. Honestly, I think it's pretty amazing that ble.sh can get the information that it does! I've seen a lot of very fancy bash scripting in my time, but I think ble.sh might take the cake :D

Thanks again for all of your help!

akinomyoga commented 3 weeks ago

Looking at fzf, my version is 0.52 (0.52.0), and the build for opensuse does not modify the man pages. I think that around this time is when fzf added native tmux support, so I guess maybe the man page did change?

I checked the behavior in Arch Linux where the fzf version is 0.52.0, but the problem doesn't reproduce. Maybe it is related to the chosen AWK implementation. Could you provide the result of the following command in your environment?

$ type ble/bin/awk

As for the rg man page, I'll later check its format and try to implement parsing of it if possible.

pallaswept commented 3 weeks ago

Thanks for looking into this for me!

> type ble/bin/awk
ble/bin/awk is a function
ble/bin/awk () 
{ 
    '/usr/bin/gawk' -v AWKTYPE=gawk "$@"
}

> /usr/bin/gawk --version
GNU Awk 5.3.0, API 4.0, PMA Avon 8-g1, (GNU MPFR 4.2.1, GNU MP 6.3.0)

If there is anything I can do to help, you got it!

akinomyoga commented 1 week ago

I added support for the man-page analysis for the rg command in commit 063bf66b1017deaad2614ee11f3358d0135ed5d6.

akinomyoga commented 1 week ago

As for the fzf completion, I confirmed the behavior in openSUSE Tumbleweed. It seems the "hyphens" in the openSUSE man page of fzf are not actually "the" hyphens, but some Unicode hyphens. You can confirm that by searching for e.g. --help in the man page of the fzf command. It doesn't hit.

This should be caused by the combination of missing escaping of hyphens in the upstream fzf man page source (fzf.1) and openSUSE's configurations of groff/nroff for the man pages. I'll later submit a pull request to the upstream fzf. I can also implement a workaround in ble.sh later because I anticipate other man pages may have the same problem of missing escaping of hyphens.

pallaswept commented 1 week ago

It seems the "hyphens" in the openSUSE man page of fzf are not actually "the" hyphens, but some Unicode hyphens

I imagine that was very obscure and not trivial to find, well done!

akinomyoga commented 1 week ago

https://github.com/junegunn/fzf/pull/3885 merged in the upstream.

akinomyoga commented 4 days ago

I pushed a workaround f160b8f02a13238fd38a546f1070a1d63ce251ce in the master branch.

pallaswept commented 4 days ago

Thanks!