taviso / wpunix

WordPerfect for UNIX Character Terminals
649 stars 18 forks source link

tsquotes.wpm inserts wrong characters #56

Closed dialognewmedia-mws closed 5 days ago

dialognewmedia-mws commented 2 months ago

It looks like the Unicode conversion in the compiled tsquotes.wpm results in an incorrect WordPerfect characters for the single quotation marks. This causes problems when the saved document is opened in other document processors, including other versions of WordPerfect.

A single left quotation mark in WordPerfect should be in set 4, character 29. Instead, tsquotes.wpm inserts a character from set 4, character 27. They look very similar but are not the same. When the file is opened in LibreOffice, it is converted into a phonetic control character. When the character is pasted into an X11 app, it becomes Single High-Reversed-9 Quotation Mark (U+201B) instead of Left Single Quotation Mark (U+2018). This happens on Windows as well when used with WSL.

Likewise, a single right quotation mark should be from set 4, character 28. Tsquotes.wpm inserts a character from set 2, character 19. WordPerfect 6+ on all platforms converts this character to 2,51 which is "˞" -- Modifier Letter Rhotic Hook (U+02DE). The character seems to paste properly on X11 and on Windows.

taviso commented 2 months ago

Hmm, there's definitely a mistake in there, maybe I was trying to determine if {2,11} {2,19} or {4,28} {4,29} were the correct pairs and somehow managed to get both wrong 🤣

It looks like I decided libwpd mapped U+2019 and U+2018 to {2,11} {2,19}, but you think {4,28} {4,29} is more correct.. hmm, I guess I will have to look at their tables.

dialognewmedia-mws commented 2 months ago

Honestly, your macro is more correct than the others I have seen floating around, some of which use the backtick character as an ersatz left single quote.

The QuickCorrect feature of WP6.x for its quotation replacement function by default lists 4,29 as left single quote; 4,28 as right single quote, 4,32 as left double quote, and 4,31 as right double quote.

And if you export or paste these characters into other applications, they are correctly rendered as the proper Unicode characters.

dialognewmedia-mws commented 2 months ago

57 pull request to address this issue.

dialognewmedia-mws commented 2 months ago

58 pull request should hopefully complete the resolution of this issue

emendelson commented 2 months ago

Do these WPDOS macros work in WPUnix?

https://mendelson.org/wpdos/addfeatures.html#smquote51

dialognewmedia-mws commented 2 months ago

These two files don't seem to work in WPUnix. I could not get them to work in my DOS WP 5.1+ copy I use for debugging either. Neither provides any output for me.

If I could get them to run, however, the file SMQUOSNG.WPM does not insert the WP characters {4,28} or {4,29}. It instead uses {`} Grave Accent (U+0060) and {'} which is Apostrophe (U+0027)

The file SMQUODBL.WPM would insert the correct {4,31} and {4,32}, if I could get it to function.

The macros from Tavis do run in my 5.1+ DOS version.

emendelson commented 2 months ago

I've never looked at those macros and I'm not surprised that they have the wrong characters. But have you assigned them to a keyboard, as the web page suggests?

dialognewmedia-mws commented 2 months ago

Yes I have assigned them, but they don't work on 5.1+ for me. They also don't work if you manually execute them as well.

Perhaps they might work on 5.1, but I don't have that around to check. Tavis's macros are almost 100%, should be working once my second pull request is confirmed, I think

emendelson commented 2 months ago

This keyboard layout seems to work perfectly in wp under WSL:

https://mendelson.org/wpdos/smartquo.wpk

It works in WPDOS 5.1+ and in wp under WSL in Ubuntu.

dialognewmedia-mws commented 2 months ago

Thanks! Confirmed that keyboard does work. I exported the saved macros in the .wpk to plaintext and then back into separate .wpm files for reassignment in my existing keyboard.

Hopefully the second pull request I mentioned above will solve the issue for people on the main branch.

emendelson commented 2 months ago

And I've now updated the single-quote macro on my web site to use the correct characters.