Shortcuts with newlines

[psads-git]

Dear All,

Is it possible to expand a shortcut with a text that contains newlines?

Example:

br

expanding to

"Best regards,

Paul"

Thanks in advance,

Paul

[mike-fabian]

I'll try to make that possible soon, it's on my TODO list.

[psads-git]

Thanks, Mike! That would be wonderful!

[mike-fabian]

Ah, one thing I am still thinking about in how to implement this is how to display multiline expansions in the candidate list.

Let's say you type “br” and this expands to

"Best regards,

Paul"

Then this expansion can occupy only one slot in the candidate list, so it could look like this:

1 | Best regards\n\nPaul 2 | brain 3 | bridge ...

or like this:

1	Best regards
Paul
2	brain
3	bridge

...

Which of these looks better?

Problems with that:

If the expansion is very long, this will look terrible, no matter whether it is displayed in one line or multiple lines in the candidate list. So maybe for very long expansions, an abbreviation is needed.

How to best show the abbreviation?

1 | Start of expansion ... end of expansion

I.e. show the first 20 characters and the last 20 characters and insert ... in between

or

1 | Start of expansion more stuff ...

I.e. Show the first 40 characters of the expansion and they cut and append a ...

or

1 | First line of expansion | ... | last line of expansioin

and all the lines between the first and the last line are collapsed to ...

What do you think would look best?

I must experiment with that and see how that feels when using it.

I tend to think that somehow abbreviating it into one line would look best, multi-line candidates look terrible in some desktop environments.

Maybe something like this would look OK:

1 | Start of first line ... [47 lines] ... end of last line

I.e. abbreviating the first and the last line to some resonable length and showing a count of the lines not shown in between.

Do you have any better ideas?

[mike-fabian]

This is how a multiline candidate in the lookup table would look on Gnome (no matter whether Gnome Wayland or Gnome Xorg):

And this is how it would look like on XFCE (and most other non-Gnome desktops):

[mike-fabian]

The display on Gnome might be sort of acceptable, but even that would look bad if the expansion is much longer.

On non-Gnome desktops like XFCE, multiline candidates are really terrible. There, the vertically tallest candidate forces all other candidates to be equally tall.

[mike-fabian]

This is how a multi line candidate would look like on Gnome when the option for a horizontal candidate list is used:

[psads-git]

Well, Mike, any user that uses shortcuts does not need to preview those in the suggestions lists. Thus,

Shortcut br

in the suggestions list would suffice. Do not you agree?

[psads-git]

Moreover, Mike, the user may want to have a shortcut for several paragraphs.

[mike-fabian]

But then one would need to remember the expansion of the shortcuts. For example, if you have a shortcut "brp" expanding to

Best regards,

Paul

and another one "brm" expanding to

Best regards,

Mike

then typing the first 2 letters gives you both shortcuts. If you see only "brp" and "brm" I find it hard to remember after a while which was which. But maybe I could get used to this.

Or maybe a combination of both, i.e. display something like:

brp ➡️ Best reg ... Paul brm ➡️ Best reg ... Mike

I.e. if one sees some text followed by ➡️ then the text before the ➡️ is the shortcut and the text behind the ➡️ gives some hint what it expands to.

How about that? Using an emoji like ➡️ saves some space compared to using a word like "Shortcut" and also makes it more stand out. And if the shortcut "brp" and "brm" is shown before the ➡️ then maybe showing a radical abbreviation of the expansion is enough as a reminder what the expansion is.

Worring about how to display such multiline expansions of shortcuts so that it looks good and understandable was the main reason why I didn’t implement this yet.

It could be useful if one needs the same blocks of text very often, it might be useful to have such shortcuts expand to very long texts, maybe whole paragraphs. Then displaying all of it becomes impossible. But maybe the shortcut, a ➡️, and then an extremely abbreviated preview of the expansion might be good?

[mike-fabian]

Currently I use some shortcuts which are only a single line but about 80 characters long. It already looks quite weird when the candidate list becomes so wide that it is cut off by the size of the screen.

[psads-git]

I like your emoji solution, Mike!

Where I use shortcuts is for writing latex: Sometimes, the shortcut involves a lot of latex commands.

Your ibus-typing-booster already provides multilined shortcuts, if one pastes those from gedit. However, for some strange reason, it does not work with Thunderbird: the blank line between is cut.

[mike-fabian]

Yes, pasting from gedit works. The only work for me to do is:

1) Make it look better in the setup tool, i.e. make the expansion a Gtk.TextBuffer() (multiple lines) instead of a Gtk.Entry() (single line) and make it scrollable

2) Make it display as nice a possible in the candidate list.

Maybe I can first do 1) and do 2) later.

Further problems are that if one has a lot of multiline shortcuts

• searching already defined shortcuts, maybe because one wants to delete some of them again becomes difficult, scrolling in a long list of multiline shortcuts is very tedious, maybe a search function is needed then.
• If one has huge amounts of shortcuts starting with many different letters, this will get in the way of completion of regular words. To avoid this, I start my shortcuts with an "ms" prefix because "ms" doesn't expand to anything in the languages I use. So I would use 'msbrp" for "Best regards, \n\n Paul". Maybe there is a better solution for that. Maybe allowing also some special letters at the beginning of such shortcuts. I am not sure. Currently, using something like \brp as a shortcut won’t work because most non-letter characters are cut away before searching for a completion. This is to make typing something like “(te” give expansions like “(testing” etc. when looking up in the dictionaries. Maybe using a short prefix like I do with “ms” common to all my shortcuts is good enough already? Or is there something I could do to improve that?

As you have found out by pasting from gedit, the basic functionality of multi-line shortcuts is already there, the work which needs to be done is improve the usability of this.

[mike-fabian]

Your ibus-typing-booster already provides multilined shortcuts, if one pastes those from gedit. However, for some strange reason, it does not work with Thunderbird: the blank line between is cut.

That is really weird. I have never used Thunderbird so far. Only Thunderbird or did you find other such cases?

[psads-git]

In GMail in Firefox, it is still worse: No newline is inserted, Mike. However, in Chrome, no problem at all!

[mike-fabian]

In GMail in Firefox, it is still worse: No newline is inserted, Mike. However, in Chrome, no problem at all!

I can reproduce that. When committing a multi-line text in GMail in Firefox, all new lines are stripped. When doing the same in GMail in Chrome, it works, the newlines are correctly inserted.

Quite weird, It doesn't happen in all input fields in Firefox. For example here when writing a github comment I can commit a multi-line text like

Best regards,

Mike

without problems.

I have no idea what could cause this, but it is certainly not the fault of ibus-typing-booster, ibus-typing-booster doesn't know whether one types in Chrome, Firefox, Github,...

In all the following 3 cases:

• Writing a new mail with GMail in Chrome
• Writing a new mail with GMail in Firefox
• Writing a github comment in Firefox

I see in the debug log of ibus-typing-booster

2021-03-09 18:51:45,923 hunspell_table.py line 5395 do_set_content_type DEBUG: purpose=0 hints=0000000000000000
2021-03-09 18:51:45,923 hunspell_table.py line 5402 do_set_content_type DEBUG: self._input_purpose = 0 (InputPurpose.FREE_FORM)

So no input hints are set and the input purpose is FREE_FORM in all 3 cases.

As it makes no difference for ibus-typing-booster, the widget to edit a new mail in GMail in Firefox must be doing something special with multi-line commits from input methods.

Nothing I can do to fix that, I think.

Bug in GMail or in Firefox I guess.

[mike-fabian]

So I think I cannot do anything about the problem in Thunderbird either, ibus-typing-booster just calls

super().commit_text(IBus.Text.new_from_string(commit_phrase))

(here: https://github.com/mike-fabian/ibus-typing-booster/blob/master/engine/hunspell_table.py#L2496 )

Where commit_phrase is a string which might contain anything, also newlines.

What happens with that string then is out of control of ibus-typing-booster. It calls a commit function in ibus and ibus then commits that string. But I don't think ibus can do anything about that either. It is mysterious ☹.

[psads-git]

Thanks, Mike. I would suggest to commit line by line and not all lines at once. It may help.

[mike-fabian]

I'll experiment with that but that may be problematic as well. Many small commits in a short time seem to have the effect that some commits get lost. I am investigating what goes wrong with ibus-m17n when using Gnome Wayland for example in this bug:

https://bugzilla.redhat.com/show_bug.cgi?id=1850832

When using these gu-itrans and hi-itrans methods with ibus-typing-booster, they work fine. Ibus-typing-booster waits until a word is finished and then commits the whole word. ibus-m17n commits as soon as one "unit" of characters is finished, this means it commits every 1-3 characters. And the some commits get lost ... That happens only in Wayland, it works in Xorg.

I am investigating at the moment whether I can work around that problem in ibus-m17n somehow, without success so far ...

When committing line by line there are two possibilities how to do it,

1) commit each line separately and between each line commit a string which only has a newline and nothing else

2) commit each line separately and between each line use forward_key_event() to forward a Return key to the application.

2) tends to work better in my experience, but forward_key_event() is often problematic in Gnome Wayland as well. In some versions of Qt it doesn't work at all (That's why ibus-typing-booster has this weird option "Use workaround for a but in Qt im module", enabling this option tries to avoid using forward_key_event() as much as possible.

Right now, in F34 Gnome Wayland, forward_key_event() seems broken again:

https://bugzilla.redhat.com/show_bug.cgi?id=1936774

[mike-fabian]

I just tried this:

diff --git a/engine/hunspell_table.py b/engine/hunspell_table.py
index de13d95..0a22435 100644
--- a/engine/hunspell_table.py
+++ b/engine/hunspell_table.py
@@ -2493,8 +2493,9 @@ class TypingBoosterEngine(IBus.Engine):
                             'surrounding_text = '
                             '[text = "%s", cursor_pos = %s, anchor_pos = %s]',
                             text, cursor_pos, anchor_pos)
-        super().commit_text(
-            IBus.Text.new_from_string(commit_phrase))
+        for commit_line in commit_phrase.splitlines(keepends=True):
+            super().commit_text(
+                IBus.Text.new_from_string(commit_line))
         self._clear_input_and_update_ui()
         self._commit_happened_after_focus_in = True
         if self._off_the_record or self._hide_input:

and it does not help in Gmail in firefox.

When the commit contains 'Best regards,\n\n\n\nMike' then this commits the following 4 chunks:

>>> 'Best regards,\n\n\n\nMike'.splitlines(keepends=True)
['Best regards,\n', '\n', '\n', '\n', 'Mike']

So one should see 3 empty lines between “Best regards,” and “Mike”. In gedit this works but in gmail in firefox one sees 4 spaces between “Best regards,” and “Mike”. Looks like this gmail in firefox thing converts every newline in a commit string into a space, no matter whether the whole thing is committed at once or in chunks.

That leaves us with forward_key_event() ...

[mike-fabian]

Now I tried the forward_key_event() idea like this:

$ git diff 
diff --git a/engine/hunspell_table.py b/engine/hunspell_table.py
index de13d95..a8d7a7a 100644
--- a/engine/hunspell_table.py
+++ b/engine/hunspell_table.py
@@ -2493,8 +2493,25 @@ class TypingBoosterEngine(IBus.Engine):
                             'surrounding_text = '
                             '[text = "%s", cursor_pos = %s, anchor_pos = %s]',
                             text, cursor_pos, anchor_pos)
-        super().commit_text(
-            IBus.Text.new_from_string(commit_phrase))
+        if self._qt_im_module_workaround:
+            super().commit_text(
+                IBus.Text.new_from_string(commit_phrase))
+        else:
+            for commit_line in commit_phrase.splitlines(keepends=True):
+                if not commit_line.endswith('\n'):
+                    super().commit_text(
+                        IBus.Text.new_from_string(commit_line))
+                    continue
+                super().commit_text(
+                    IBus.Text.new_from_string(commit_line[:-1]))
+                self.forward_key_event(
+                    IBus.KEY_Return,
+                    self._keyvals_to_keycodes.ibus_keycode(IBus.KEY_Return),
+                    0)
+                # The sleep is needed because this is racy, without the
+                # sleep it is likely that all the commits come first
+                # followed by all the forwarded Return keys:
+                time.sleep(self._ibus_event_sleep_seconds)
         self._clear_input_and_update_ui()
         self._commit_happened_after_focus_in = True
         if self._off_the_record or self._hide_input:

This works in gedit and in GMail in firefox. On my it desktop on Fedora 33.

Note that the sleep is necessary, when committing something like 'Best regards,\n\n\n\nMike' without the sleep the result in gedit and Gmail in firefox is 'Best regards,Mike\n\n\n\n', i.e. all commits come first and all forwarded Returns come after all committed strings.

With

        self._ibus_event_sleep_seconds = 0.1

The order was always right when I tested. But that might not work anymore when the system load is high, so I don't really like that hack.

But I see no alternative at the moment. At least with that hack it works in more cases than without and it does not much harm.

• When strings without newlines are committed, nothing is changed (i.e. Users who don't use multi-line shortcuts should see no difference at all in behaviour, no matter whether forward_key_event() works or not)
• When multiline shortcuts are used and forward_key_event() works, this makes ist work in GMail in Firefox (and probably in Thunderbird as well) and it still works in gedit (although you can see the slight slowdown introduced by the 0.1s sleeps between the lines).
• When multiline shortcuts are used and forward_key_event() is broken, one has to use the option “Use a workaround for a bug in Qt im module” (which I should really rename to “Avoid forwarding key events” ...) and when using that option it works as before, i.e. works in gedit but not in gmail in firefox.

[psads-git]

Thanks, Mike. I do not know what is the mechanism that AutoKey,

AutoKey ,

uses. However, multilined AutoKey's shortcuts expand correctly both in Thunderbird and GMail in Firefox. Maybe you could have a look at AutoKey for getting further ideas.

[mike-fabian]

Thanks, Mike. I do not know what is the mechanism that AutoKey,

AutoKey ,

uses. However, multilined AutoKey's shortcuts expand correctly both in Thunderbird and GMail in Firefox. Maybe you could have a look at AutoKey for getting further ideas.

Auto key does not seem to be an ibus input method, so probably it cannot help me with problems committing strings and forwarding key events via the ibus API.

[mike-fabian]

I wrote:

brp ➡️ Best reg … Paul

and you liked that idea:

I like your emoji solution, Mike!

But it looks like I cannot do that so easy without extra database lookups. Maybe later.

Until now I made only a few small changes to make it more usable with multiline shortcuts:

• If a commit string contains new-lines, commit it in several chunks and forward Return between the chunks
• If a candidate contains newlines, replace the newlines with arrows in the lookup table
• Elide extremely long candidates in the lookup table in the middle

So something like “Best regards,\n\n\nPaul” is displayed as “Best regards,↩︎↩︎↩︎Paul” in the lookup table.

And if an entry in the lookup table is longer than 80 characters only the first 40 and the last 40 characters are shown with eliding dots in the middle.

[psads-git]

That is fine, Mike! Thanks! Will a rpm of ibus-typing-booster be meanwhile created and made available to dnf? If not, could you please direct me to some documentation where I could find information about how to install ibus-typing-booster from source?

[mike-fabian]

I think you are using Fedora 33 so I build a test rpm for you here:

https://copr.fedorainfracloud.org/coprs/mfabian/ibus-typing-booster/build/2063136/

https://download.copr.fedorainfracloud.org/results/mfabian/ibus-typing-booster/fedora-33-x86_64/02063136-ibus-typing-booster/

This is not yet the "real" 2.10.6 release, just a release candidate, I might add other stuff before the release.

The setup tool is not yet modified to allow multiline shortcuts, so you still have to paste from gedit.

[mike-fabian]

With the rpm from the last comment, it looks like this:

[mike-fabian]

And I think it should work for you now in GMail in firefox, at least it worked for me with this update. And it will probably work in Thunderbird, I didn't test that, but maybe you can test that.

[psads-git]

Thanks, Mike! With the COPR version, no problem with multiline shortcuts in GMail in Firefox and in Thunderbird. Perfect!

[mike-fabian]

You can notice the small 100ms delay after each line though. If you have a shorcut which is 10 lines long, inserting that shortcut takes about a second. Unfortunately that hack with the delay is necessary to make it work. And there may be situations, maybe when the system is under very high load, that 100ms are not enough. I am not sure, for me this works reliably. I use a similar delay in ibus-typing-booster for a long time already in other places. For example, if you type a word, then use the arrow left key to go to the middle of the word (while the word is still in preedit and not committed), then type "Return". In that case, the part of the word to the left of the cursor is committed, then a Return key is forwarded, then the part of the word to the right of the cursor is forwarded. In between I also needed to insert delays to make that work reliably. But this never failed for me so far, so I think it works and the only disadvantage is that inserting shortcuts which huge expansions takes noticable time.

[psads-git]

Thanks, Mike. The delay is not noticeable at all. Great job!

[mike-fabian]

I just improved the setup tool to make it possible to write multi-line expansions there directly, without pasting from gedit.

New rpm coming in a few minutes....

[mike-fabian]

New build with the improved setup tool:

https://download.copr.fedorainfracloud.org/results/mfabian/ibus-typing-booster/fedora-33-x86_64/02063989-ibus-typing-booster/

Looks like this:

[mike-fabian]

Note that when you press the "Add shortcut" button, I remove whitespace from the beginning and the end of the text found in the buffer which contains the expansion.

I am not sure about it. I thought that especially the white space at the end might be confusing, as you cannot see it.

But removing the leading and trailing white space makes it impossible to have an expansion which adds a newline or spaces at the end.

Should I allow leading and trailing white space?

[psads-git]

The most recent version is working just fine, even in Thunderbird and GMail/Firefox. Thanks, Mike!

To me, deleting the leading and trailing white spaces is perfectly fine.

I would suggest the following improvements (not priority for now):

1. To make size-adjustable the window where the user writes the expansion text.

2. To allow the edition of saved shortcuts.

[mike-fabian]

1. To make size-adjustable the window where the user writes the expansion text.

Good idea, I may do that later as it is a bit more effort.

I did the following small improvement which was easy now:

• make the window for the expansion text vertically expandable
• make the button area at the bottom not expandable.

That way, the window for the expansion text and the window for the already defined shortcuts are the only vertically expandable things in that page, if you resize the setup tool, both will get bigger. The area for the "About" and "Close" buttons at the bottom used to get bigger when resizing the setup tool but not anymore in order not to waste useful space on these buttons.

[mike-fabian]

2. To allow the edition of saved shortcuts.

You know that you can define more than one different expansion for the same shortcut?

Your suggestion is useful though, I am thinking about how to do that best.

[mike-fabian]

Somewhat bigger expansion window and somewhat resizable by resizing the whole setup tool:

https://download.copr.fedorainfracloud.org/results/mfabian/ibus-typing-booster/fedora-33-x86_64/02067353-ibus-typing-booster/

[psads-git]

Now, Mike, it is perfect! Thanks for your efforts to make our life so much easier through ibus-typing-booster and its improvement!

[mike-fabian]

I will try to make the resizing work better eventually by making it possible to move a border between the upper and lower window up and down with the mouse. But that is difficult and I don’t know how to do it at the moment so I will not do this in the next month. But later probably, if I figure out how.

[psads-git]

That is very fine, Mike: it is an improvement of not high priority.

[mike-fabian]

To allow the edition of saved shortcuts.

Can you open a new issue for this one, so that it does not get forgotten?

[mike-fabian]

Fedora 33 build of ibus-typing-booster 2.11.0 is here:

https://download.copr.fedorainfracloud.org/results/mfabian/ibus-typing-booster/fedora-33-x86_64/02068370-ibus-typing-booster/

[psads-git]

Thanks, Mike!

[mike-fabian]

I improved the behaviour of the buttons “Clear input”, “Delete shortcut”, and “Add shortcut” buttons a bit.

Now the delete and add buttons are only sensitive when both the shortcut entry and the expansion entry contain some none-whitespace text. And the clear button is only sensitive if at least one of them is not empty (whitespace only counts as not empty).

Before the buttons were always sensitive.

[mike-fabian]

The list of already defined shortcuts at the bottom is currently ordered in the order they were defined.

I thought alphabetical might be better, not sure ...

[psads-git]

Thanks, Mike! If it does not imply too much work, it would be best if the shortcuts were (user) sortable by each field.

[psads-git]

There is something that it strange (maybe a bug): shortcuts that begin with numbers never seem to be suggested. For instance,

1a

to expand to

One artist.

[mike-fabian]

Thanks, Mike! If it does not imply too much work, it would be best if the shortcuts were (user) sortable by each field.

Can you open a new issue for that to remind me?

[mike-fabian]

There is something that it strange (maybe a bug): shortcuts that begin with numbers never seem to be suggested. For instance,

1a

to expand to

One artist.

That is not really a bug.

The reason for this is that with the default key bindingd, “1” and “KP_1” (The 1 one the number pad if your keyboard has a number pad) commit the first candidate and a space. These are the default key bindings:

[mike-fabian]

Now if “1” is used to commit a candidate you cannot type it into the preedit. For example, if you type “a” you have “a” in the preedit and see some completions starting with “a” in the candidate list. If you then type a “1” the first of these candidates is committed, you do not get “a1” in the preedit.

Let’s assume the preedit is empty, and you type a “1”. Why not put this into the preedit? There are no candidates yet, so there is no conflict with committing a candidate? So putting it into the preedit should be OK, no? No, it is not OK because as soon as the 1 is in the predit are most likely candidates and now you cannot type the next digit. So if you want to type the number eleven “11” and the first “1” went into the preedit, the second “1” would commit something and you would not get the number eleven you wanted to type.

That means, if digits like “1” are used for any keybindings (especially the key bindings to commit candidates but also some othes would be problematic), then they should not start a preedit to make it still possible to type numbers.

So if the preedit is empty and the first key is typed, and it is a number, and numbers are used in any keybindings, then ibus-typing-booster doesn’t open a preedit and puts that number into the preedit but rather commits that number directly (possibly converted to a localised number if you use Arabic or Indic input methods ...).

But there is one possibility to change this behaviour, maybe it is enough for you to use F1…F9 to commit candidates? You don’t need numbers in any keybindings at all maybe?

Then you can remove all numbers from the keybindings and then you can input them into the preedit and you can type “1a” into the preedit and it can complete or expand as a custom shortcut.

That means you could make the keybindings like this, and they you can type the numbers into the preedit: