Strange 777; preexec

[okbob]

On master I see unwanted content

[pavel@nemesis libvterm]$ export PSQL_PAGER="pspg --vertical-cursor -X -s 0" 
777;preexec [pavel@nemesis libvterm]$ 

[TragicWarrior]

@okbob , is there something special about your environment? I tested on Fedora 30 with no problems.

[bryanc@localhost libvterm]$ cat /etc/fedora-release Fedora release 30 (Thirty)

[TragicWarrior]

[okbob]

čt 17. 10. 2019 v 22:21 odesílatel TragicWarrior notifications@github.com napsal:

[image: 2019-10-17_15-17_libvterm] https://user-images.githubusercontent.com/8481559/67044370-c2f2fe00-f0f1-11e9-92ea-8aebe88530b2.png

I have not any special customization, but i have a Fedora 31 now

https://user-images.githubusercontent.com/8481559/67044370-c2f2fe00-f0f1-11e9-92ea-8aebe88530b2.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/TragicWarrior/libvterm/issues/147?email_source=notifications&email_token=AAEFO4ZM6X5PD5MXLB3JJGLQPDCMBA5CNFSM4JBTZXQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBRMXVA#issuecomment-543345620, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEFO44VBE6VNDP2AKUJ64DQPDCMBANCNFSM4JBTZXQQ .

[TragicWarrior]

@okbob , i'll have to upgrade to Fedora 31. newer versions of bash seem to be abusing OSC commands for their own proprietary use. that's what i'm guessing is going on here. they really need to stop doing that.

[TragicWarrior]

@okbob , the culprit is the environment variable PS0 which is used by Bash. The Bash documentation for it says:

PS0 The value of this parameter is expanded like PS1 and displayed by interactive shells after reading a command and before the command is executed.

By unsetting this variable, the problem goes away. I'm not sure if there's a reasonable way to handle this. Based on the little bit of reading I've done, it seems to be used for debug purposes and I wonder if it will be unset by default after Fedora 31 is released.

[okbob]

we will see. Same behave has xterm. On second hand, gnome-terminal doesn't do it.

[TragicWarrior]

Yes. The problem occurs with konsole as well. Although it's possible to read PS0 and then parse it out from the stream, it doesn't seem right to do since PS0 and PS1 are designed so that before and after a command some kinds of text / information will be emitted. In other words, to eat PS0 and PS1 would violate the purpose of them.

[TragicWarrior]

@okbob , I think this issue needs to be reported to the Fedora Bugzilla. There may be a good reason for having PS0 and PS1 set during Beta but if so, they need to explain that.

[smoebody]

@okbob did you report this already? do you have an issue-number? I am experiencing the same issue after upgrading to fedora 31 but in guake-terminal with tmux. No problem at all with gnome-terminal and tmux or guake-terminal alone.

[okbob]

út 5. 11. 2019 v 9:06 odesílatel Ulf Seltmann notifications@github.com napsal:

@okbob https://github.com/okbob did you report this already? do you have an issue-number? I am experiencing the same issue after upgrading to fedora 31 but in guake-terminal with tmux. No problem at all with gnome-terminal and tmux or guake-terminal alone.

No, I didn't do any report already. Just I know too little about this topic.

Pavel

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/TragicWarrior/libvterm/issues/147?email_source=notifications&email_token=AAEFO4YYWS7CHVHHJ345YWLQSESQVA5CNFSM4JBTZXQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDB6IZY#issuecomment-549708903, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEFO4YUKHYJHOXJLYNOVNLQSESQVANCNFSM4JBTZXQQ .

[ThomasDickey]

fwiw (just started looking at Fedora31) -

• rishi pushed to vte291 (master). "Add signals proxying an interactive shell's precmd and preexec hooks", and
• Bug 711059 - Notifications for long-running commands

The terminal programs ignore this; the text is echoed (apparently) due to a scripting error.

[TragicWarrior]

@ThomasDickey , so as I understand this, OSC 777 is now a "thing" and should be parsed out?

[ThomasDickey]

yes - it's not of much interest (tells urxvt to run a perl script), and like any unrecognized control sequence can be ignored.

[TragicWarrior]

@okbob , I'll try to make a change next week that discards OSC 777.

[LinuxGeek46]

I noticed the following anomaly in non-root screen sessions:

[dboth@david ~]$ echo "Hello world" 777;preexecHello world [dboth@david ~]$

The extraneous 777;preexec text is inserted in PS1 by the /etc/profile.d/vte.sh script. Commenting out the last few lines, the “case” statement circumvents the problem.

This problem does not appear in root screen sessions or in any non-screen sessions, root or otherwise.

I hope this helps.

There is bugzilla #1780785 on this problem

[TragicWarrior]

@LinuxGeek46 , you are correct. for a short term fix that will do. however, as @ThomasDickey noted above, OSC 777 is a new sequence that must be handled. Lord willing, i will try to accommodate it in the next couple of weeks.

[LinuxGeek46]

Please let me know if I can help. I will be glad to test your patch.

[lwi]

In my tests and by reading the source only the 8bit OSC identifier is not handled. If the prompt string would be something like "\033]777;preexec\a" it should be properly handled. Since I don't have Fedora I guess the string is "\235777;preexec\234" which includes a 8bit OSC identifier. Could someone confirm?

[LinuxGeek46]

This is the line in /etc/profile/vte.sh that I comment out to circumvent the issue. If you need more information, please let me know.

[ -n "$BASH_VERSION" ] && PROMPT_COMMAND="__vte_prompt_command" && PS0=$(printf "\u009D777;preexec\u009C")

On 12/10/19 11:38 AM, Leon Winter wrote:

In my tests and by reading the source only the 8bit OSC identifier is not handled. If the prompt string would be something like "\033]777;preexec\a" it should be properly handled. Since I don't have Fedora I guess the string is "\233777;preexec\234" which includes a 8bit OSC identifier. Could someone confirm?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/TragicWarrior/libvterm/issues/147?email_source=notifications&email_token=AIKYCQ3CHAPDFCOU4L2PHXTQX7AYTA5CNFSM4JBTZXQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGP4S7I#issuecomment-564119933, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIKYCQ6J5HQ5DZ5LTBDAK23QX7AYTANCNFSM4JBTZXQQ.

--

---

David P. Both

---

The value of any software lies in its usefulness not in its price.

— Linus Torvalds

---

[lwi]

@LinuxGeek46 perfect, this confirms my suspicion. Implementing C1 8-bit however involves touching a lot of places since the codebase unfortuantely uses signed chars everywhere. I am looking into it.

[lwi]

There is another far more serious obstable, C1 8-bit codes have values that are also valid UTF-8 continuation characters. However they are not in the range of UTF-8 starting characters so they could safely be determined. On the other hand if one wanted to use a cool feature of UTF-8, namely (self) synchronization, one could not tell whether the byte at hand is meant as control character or continuation byte. That being said, libvterm does not use this feature but will instead discard characters up to UTF8_BUF_SIZE. My approach now is to implement a correct UTF-8 start detection, which would enable me to tell them apart from C1 characters. An hackish attempt at the C1 characters would otherwise break UTF-8, which probably noone wants.

[lwi]

PoC C1 handling, actually without proper UTF-8 detection, try this: https://github.com/lwi/libvterm/tree/fix-osc-777

[lwi]

@LinuxGeek46 @okbob any luck testing it?

[LinuxGeek46]

I have not seen anything to test. Where can I d/l it?

On 12/12/19 8:56 AM, Leon Winter wrote:

@LinuxGeek46 https://github.com/LinuxGeek46 @okbob https://github.com/okbob any luck testing it?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/TragicWarrior/libvterm/issues/147?email_source=notifications&email_token=AIKYCQ32PZPH6U77IRZTFVLQYI7JVA5CNFSM4JBTZXQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGWXTGA#issuecomment-565016984, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIKYCQ6OUUL3BQBF52THDJDQYI7JVANCNFSM4JBTZXQQ.

--

---

David P. Both

---

The value of any software lies in its usefulness not in its price.

— Linus Torvalds

---

[okbob]

I tested your repository - just I use it in last Fedora, and this issue is there still

[danmack]

I am also noticing this issue when using the Alacritty (https://github.com/jwilm/alacritty) terminal emulator. I've just commented out the case block as mentioned above which is my current work-around. I am on an up-to-date version of fedora 31 workstation.

[lwi]

I tested your repository - just I use it in last Fedora, and this issue is there still

What is the exact sequence that you feed to libvterm? It is quite hard to debug this without knowing that.

[lwi]

Nevermind. I revisited the test case and found that it is even more complicated. The codepoint for 8bit C1 control character itself is encoded in UTF-8. I am not sure this is allowed. Why didn't they use the 7bit sequence (also with length of 2 bytes) instead? Anyway, I will get back to you with a fix once I figured out it whether and how should be supported.

[lwi]

Sadly unlike xtem some terminal emulators support UTF-8 encoded C1 sequences. I am undecided what do to. @TragicWarrior if we want to support C1 sequences inside UTF-8 we would need to change the code somewhat, parsing the decoded unicode points for such control sequences.

[lwi]

@okbob should work now @LinuxGeek46 you can test my branch https://github.com/lwi/libvterm/tree/fix-osc-777 like this: git clone -b fix-osc-777 --depth 1 https://github.com/lwi/libvterm;cd libvterm;cmake .;make;LD_LIBRARY_PATH=. ./vshell

[okbob]

ne 15. 12. 2019 v 18:33 odesílatel Leon Winter notifications@github.com napsal:

@okbob https://github.com/okbob should work now @LinuxGeek46 https://github.com/LinuxGeek46 you can test my branch https://github.com/lwi/libvterm/tree/fix-osc-777 like this: git clone -b fix-osc-777 --depth 1 https://github.com/lwi/libvterm;cd libvterm;cmake .;make;LD_LIBRARY_PATH=. ./vshell

it is fixed

Pavel

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/TragicWarrior/libvterm/issues/147?email_source=notifications&email_token=AAEFO47BNUIDJ224ICTV36DQYZS43A5CNFSM4JBTZXQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG4575A#issuecomment-565829620, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEFO4Y55QSUKESTORZIU5LQYZS43ANCNFSM4JBTZXQQ .

[TragicWarrior]

@lwi , do you intend to create a pull request for "fix-osc-777" or is that just a stop gap lieu of something else?

[lwi]

@TragicWarrior with the pull request done, so you can close this bug

[TragicWarrior]

@lwi , thanks!