Problems with non-us keyboards (German Swiss German)

[marcoxa]

Describe the bug It appears that the latest version (on W11 WSL Ubuntu 24.04) does not like a Swiss German keyboard. I cannot type in the top left key (§ and ° when shifted) below the Escape key and, above all, the bottom left key next to the left SHIFT key (> when shifted and Alt Gr \ work correctly, but not the plain <; which is a problem).

Other keys do not quite work. Some of them produce accented characters (ü. à. ö etc) but in general it is a bit annoying).

To Reproduce Steps to reproduce the behavior:

1. W11
2. Swiss German keyboard

Expected behavior I press < I get a <.

Context (please complete the following information):

• OS: W11 WSL Ubuntu 24.04
• Host arch: I7
• IL:MAKESYSDATE: 26-Jun-2024 02:24:10

[pamoroso]

Did earlier Medley versions work better with the Swiss German keyboard? There are longstanding known issues with and incomplete support of non English keyboards, for example #334.

[marcoxa]

Not sure @pamoroso ... I had an earlier version installed, but it did not start anymore. So I just deleted it. Sorry.

[marcoxa]

... plus this is on Windows 11 with VNC; not MacOS. It must be something with (Swiss) German Keyboards :)

[masinter]

see also problems with Italian keyboard at https://github.com/orgs/Interlisp/discussions/1828

[rochus-keller]

Some of them produce accented characters (ü. à. ö etc)

@ marcoxa: did you really manage to make Medley display an "Umlaut" character? Do you remember what key strokes you had to type for this?

[marcoxa]

HI.

no. I actually had more basic problems, like, ahem, open parentheses :)

MA

On Wed, Oct 9, 2024 at 2:47 PM Rochus Keller @.***> wrote:

Some of them produce accented characters (ü. à. ö etc)

@ marcoxa: did you really manage to make Medley display an "Umlaut" character? Do you remember what key strokes you had to type for this?

— Reply to this email directly, view it on GitHub https://github.com/Interlisp/medley/issues/1780#issuecomment-2402227741, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD5SWQGKHR44LGY475OAH3Z2UQXDAVCNFSM6AAAAABKNIGB56VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBSGIZDONZUGE . You are receiving this because you authored the thread.Message ID: @.***>

-- Marco Antoniotti Somewhere over the Rainbow

[pamoroso]

The Italian keyboard has similar issues. But a good workaround is to run Medley under VNC (via medley -v +), which lets me at least type all (except <) the ASCII characters on the Italian keyboard including (, ), ;, and ;.

[rochus-keller]

But a good workaround is to run Medley under VNC

Because VNC makes a keyboard/key sequence translation to an ASCII keyboard? After looking at the sdl.c code I think we can have this cheaper, without VNC.

[pamoroso]

I actually don't know why VNC does the right thing.

[rochus-keller]

On what client machine are you working? PC Linux, PC Windows, Mac?

[pamoroso]

I'm working on a Linux Mint Cinnamon PC.

[masinter]

the issues listed in interlisp keyboard 'project' have more details scattered in the discussions of the issues.

[rochus-keller]

@pamoroso : meanwhile I implemented a solution in https://github.com/rochus-keller/Gingko which should work with any keyboard; just download the repository to your machine and then run gcc *.c -std=c99 -lSDL2 -lm -o gingko. Then set the MEDLEYDIR environment variable (export MEDLEYDIR=<path-to-medley-directory>) where the medley directory is the one in the standard deployment (i.e. on the same level as the maiko and notecards directories) and run ./run-medley. Just ask if you need help. I did some tests, but I'm neither an Interlisp nor Medley user, so I would appreciate if you could check whether everything works as expected.

@marcoxa : maybe you also want to give it a try.

[marcoxa]

Thanks. I don’t have much time now but I will keep you posted. Note that I am on Windows though.

MA

Marco Antoniotti Somewhere over the Rainbow

On Wed, 9 Oct 2024 at 23:24, Rochus Keller @.***> wrote:

@pamoroso https://github.com/pamoroso : meanwhile I implemented a solution in https://github.com/rochus-keller/Gingko which should work with any keyboard; just download the repository to your machine and then run gcc *.c -std=c99 -lSDL2 -lm -o gingko. Then set the MEDLEYDIR environment variable (export MEDLEYDIR=) where the medley directory is the one in the standard deployment (i.e. on the same level as the maiko and notecards directories) and run ./run-medley. Just ask if you need help. I did some tests, but I'm neither an Interlisp nor Medley user, so I would appreciate if you could check whether everything works as expected.

@marcoxa https://github.com/marcoxa : maybe you also want to give it a try.

— Reply to this email directly, view it on GitHub https://github.com/Interlisp/medley/issues/1780#issuecomment-2403451007, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD5SWSE54RIEOHOOBISRNTZ2WNKLAVCNFSM6AAAAABKNIGB56VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBTGQ2TCMBQG4 . You are receiving this because you were mentioned.Message ID: @.***>

[pamoroso]

@rochus-keller Thanks, I'll test your code by the end of the week.

[nbriggs]

The library/virtualkeyboards/VIRTUALKEYBOARDS package, with XKEYBOARDS, has definitions for Italian and German (amongst other) keyboards - can someone with those physical keyboards try this out (under X, of course). It will (should) fix the cases where the keycap on the non-US keyboard differs from the US keycap for the same physical position code (which is what Lisp works in, and Maiko attempts to support). p.s., It also appears to support a Dvorak layout - I don't know if we have any intersection between Dvorak users and Medley users, but apparently there was once at least one.

[rochus-keller]

@pamoroso : the most recent version can even be used without setting any environment variables or shell scripts; just build it (gcc *.c -std=c99 -lSDL2 -lm -o gingko) and give it the path to the required sysout file in the medley directory (./gingko -sysout <path-to-sysout>, or even simpler ./gingko <path-to-sysout>); and it can still be used as a drop-in replacement of ldesdl in the original installation.

[pamoroso]

I built the latest sources of Gingko on my Linux Mint 22 Cinnamon amd64 PC with this Italian keyboard (see the photo).

Typing all the ASCII characters works fine with the keys by the same symbols, including those on the numeric keypad except for ENTER which does nothing. The Ctrl-U and Ctrl-X keychords don't work either and pressing them does nothing.

Great job!

[pamoroso]

Ctrl-B, Ctrl-D, and Ctrl-E don't work either with none of the two Ctrl keys. Pressing the left Ctrl alone does nothing, pressing the right one flashes Medley.

[pamoroso]

The Tab, Shift, Caps Lock, and Meta keys work as expected.

[rochus-keller]

Thanks for your tests and feedback! Are you using the stand-alone gingko or as a drop-in replacement of ldesdl? What are the CTRL+key combinations supposed to do? In the Medley Language Reference I found CTRL + mouse buttons or CTRL+SHIFT stand-alone but wasn't able to test it.

[pamoroso]

I'm using Ginko standalone built it under ~/medley/Gingko. To use it I set MEDLEYDIR, cd to ~/medley/Gingko, and execute ./run-medley -e -n. Without MEDLEYDIR I get this error:

paolo@lispmachine:~/medley/Gingko$ ./run-medley 
MEDLEYDIR has no loadups: /home/paolo/medley/Gingko
I tried to infer it based on your working directory, but that didn't work.
Try cd there or setting the MEDLEYDIR environment variable to its location.

Here's what the keychords do on Medley:

• Ctrl-U: same as in a Unix shell, i.e. deletes the current line in an Exec
• Ctrl-X: moves the caret to the end of a s-exp in an Exec and evaluates it (if it's complete)
• Ctrl-B: invokes the debugger
• Ctrl-D, Ctrl-E, different ways of interrupting the current program

[rochus-keller]

I tried to infer it based on your working directory, but that didn't work.

If you run gingko just with the path to a sysout, it looks whether the path includes "/loadups/" and then (and only then) sets MEDLEYDIR to the path segment left of "/loadups/", LDESOURCESYSOUT to the full path passed to gingko, INMEDLEY to 1, LDEDESTSYSOUT to "${HOME}/lisp.virtualmem", and LDEINIT to "/greetfiles/MEDLEYDIR-INIT" in MEDLEYDIR. So it actually should be able to run from any directory, without the shell script.

Thanks for the shortcut descriptions. I will try to make it work.

[pamoroso]

Passing a sysout doesn't seem to work either, here's with MEDLEYDIR not set:

paolo@lispmachine:~/medley/Gingko$ ./run-medley ~/medley/medley/loadups/full.sysout 
MEDLEYDIR has no loadups: /home/paolo/medley/Gingko
I tried to infer it based on your working directory, but that didn't work.
Try cd there or setting the MEDLEYDIR environment variable to its location.

[rochus-keller]

Can you try "./gingko <absolute-path-to>/full.sysout", i.e. without the run-medley shell script and with the absolute path?

Note that if you start with the shell-script (run-medley) instead, you have to manually set MEDLEYDIR, as described here: https://github.com/rochus-keller/gingko/#status-on-2024-10-09

[pamoroso]

Yes, calling gingko with the absolute path of a sysout does work:

paolo@lispmachine:~/medley/Gingko$ ./gingko ~/medley/medley/loadups/full.sysout 
Failed to find UNIXCOMM file handles; no processes
requested width: 1024, height: 768
initialised
Window created
Creating renderer...
Creating texture...
SDL initialised

[rochus-keller]

Ok, great. ./gingko -h shows the available options, of which I have tested -sc and -sysout so far. Note that you can either set the sysout path using the -sysout option or directly without option; both variants do the same; in the latter case, the options must follow the sysout path.

[nbriggs]

Here's what the keychords do on Medley:

Ctrl-U: same as in a Unix shell, i.e. deletes the current line in an Exec Ctrl-X: moves the caret to the end of a s-exp in an Exec and evaluates it (if it's complete) Ctrl-B: invokes the debugger Ctrl-D, Ctrl-E, different ways of interrupting the current program

Hold on a moment there - that is NOT "what the keychords do on Medley" -- that's how the default TTYIN reader handles some control characters, unless the user has changed them -- the interpretation is NOT in the virtual machine. The physical key down/up transitions are passed to Medley, which then runs them through the user/application settable key action tables in order to determine how they are interpreted. The emulator is NOT involved.

[rochus-keller]

@pamoroso : I just commited a new version of Gingko which supports CTRL+letter combinations, als well as the Enter key; the reason it didn't work is that SDL apparently doesn't send a SDL_TEXTINPUT message to the application if the CTRL key is pressed, so I had to extend the behaviour of SDL_EVENT_KEY_DOWN. I also added terminal messages if gingko sets the MEDLEYDIR environment variable, so you know what you're up to. Hope everything works now.

@nbriggs : Thanks for the clarification; I assumed so and pass the shortcuts to Lisp to handle.

[nbriggs]

What happens if the user swaps the shift and control key actions in Lisp? e.g.,

(DEFINEQ (SWAP-SHIFT ()
    (LET ((OSHIFT (KEYACTION 'LSHIFT))
         ((OCTRL (KEYACTION 'CTRL)))
      (KEYACTION 'LSHIFT OCTRL)
      (KEYACTION 'CTRL OSHIFT)))

(or, more likely, swaps the shift-lock key and control key)

[rochus-keller]

To avoid interference with the window manager of the client, such modifications should be done on OS level (e.g. by xmodmap on Linux). To a certain degree it would be possible with Maiko or Gingko, though not all key strokes are routed to the application anyway. Such issues are common with virtual machines or remote receivers, where there is a host and a remote OS (Medley takes some OS repsonsibilities, as e.g. Smalltalk did).

[nbriggs]

I wouldn't characterize it as working "to a certain degree" under X11 - it works as expected, independent of whether the display is local or remote. The window manager shouldn't/doesn't get involved if the input focus is in the application window.

[rochus-keller]

Unfortunately the X11 version didn't work on my Linux i386 machine. I managed to compile it, but the display was distorted, so I had to use SDL instead, where at least the display worked, and now also the keyboard works good enough for me to continue my exploration.

[nbriggs]

How was the display distorted? Which Linux? It has run on every X11 we've encountered over 30 or so years without running into this issue so I'd like to dig into what went wrong here.

[rochus-keller]

That's how it looks with the original X version of Maiko:

I'm using a 2009 EliteBook 2530p with Linux Mint Rebecca for this kind of work. But don't worry, everything is fine for me now. Maybe some European X version users would welcome my keyboard fixes, but of course it's up to the project team to decide what solution should be preferred.

[nbriggs]

Hmm... that's very peculiar. It looks like an X11 image update issue - as if it was miscalculating the pitch of the area being updated, which would be internal to the X library code. But, if you start it with (from the medley directory, with the X11 lde and ldex built):

./run-medley   -g 1024x700 -sc 1024x700 -noscroll internal/loadups/starter.sysout

does it do the same thing?

[pamoroso]

With the new version of Gingko the Ctrl keychords work only part ot the time. For example, sometimes I type text at an Exec, delete the text with Ctrl-U, immediately type new text on the same line, press Ctrl-U again, and nothing happens. Similarly for other Ctrl keychords.

An issue that wasn't present with the previous version is that now the system seems to miss some keypresses not involving modifiers. For example, sometimes I type REDO at an Exec (I'm sure I press all the keys) but only RED is displayed.

The ENTER key of the numeric keypad still does nothing.

@nbriggs calling SWAP-SHIFT with the new version of Gingko yields the error (OCTRL (KEYACTION (QUOTE CTRL))) is not a SYMBOL. as in this screenshot:

I had to type in your code because the clipboard doesn't work (I didn't try with the previous version of Gingko). Pasting text into Medley with Meta-V at an Exec yields this error at the Linux shell:

reading UnixPipeIn: Bad file descriptor

then the Exec displays a blank line and the next prompt.

By the way, typing < works fine with Gingko.

[nbriggs]

@pamoroso - did you try to start the Gingko ldesdl directly or by putting it where an lde built for SDL would find it? You can't do anything that relies on shell calls to Unix if you run ldesdl (or ldex) directly. I think I typed an extra ( in ((OCTRL. That's clearly not a valid LET expression!

[rochus-keller]

@nbriggs : I start the application with ./run-medley --dimensions 1200x768 -noscroll. Here is how the screen looks immediately after start, without any interactions:

It doesn't look like an update issue for me; also the color doesn't seem to be correct and the fan of my laptop turns high:

e.g. if I press the left most top menu, I get this:

if I then select the next top menu, I get this:

The issue seems to be very selective. But as I said, don't worry. Since there is not precompiled version of x86 and I read somewhere that only x86_64 is supported, I thought that there might be some 32/64bit issue. But it works with SDL which is ok for me.

@pamoroso : thanks for the feedback; I will review the code whether there could be some logic error causing this pattern. Concerning the pipe, please keep in mind that when directly running ldesdl, ldex or gingko (instead of running it indirectly via lde) the process feature and with it some communication pipes are not there, which might be the reason for the observed behaviour. Since I'm not familiar enough (yet) with Medley I unfortunately have no experience how the system should react in normal case.

[nbriggs]

@rochus-keller - I am not sure where you'd have read "only x86_64 is supported" (if it's on interlisp.org or in our github I'll fix it!) - I compile and run it on an x86 32-bit platform regularly.

FreeBSD 14.1-RELEASE-p3 GENERIC i386
FreeBSD clang version 18.1.5 (https://github.com/llvm/llvm-project.git llvmorg-18.1.5-0-g617a15a9eac9)
WARNING: 32-bit kernels are deprecated and may be removed in FreeBSD 15.0.
CPU: Intel(R) Pentium(R) M processor 1.70GHz (828.13-MHz 686-class CPU)

The colors should be pure black and pure white. Could you try it with the run-medley options I suggested above? If it doesn't start up with a clean (unscrambled) screen there's no need to go further, it won't get better.

[rochus-keller]

Ok, here is what I get using the commandline you proposed:

[nbriggs]

Do you know where I can get an .iso installer image for the version of Linux Mint you're running? (17.1/Rebecca is not on the mirror sites due to its age) I'll boot it up on a 32-bit VirtualBox on my Mac and see whether it's repeatable.

[pamoroso]

To run Gingko I connect to its source directory and execute ./gingko ~/path/to/full.sysout. The oly other lde* programs are in the path from my regular Medley build.

With the fixed SWAP-SHIFT code the Shift equivalent of Ctrl-U works reliably all the times, as well as the equivalents ofCtrl-B, Ctrl-D, and Ctrl-E.

But for some characters I normally type shifted, such as Shift-8 for ( and Shift-9 for ), the Ctrl equivalents produce nothing.

For other shifted symbols I get something different, for example ^V when typing Ctrl-ì which I normally get with Shift-i.

[rochus-keller]

@nbriggs : Google revealed that there is a downloadable version at the internet archive (https://archive.org/details/linuxmint-17.1-cinnamon-32bit.iso_201709), though the site seems to be down and I'm not sure whether they have the mate edition which I'm using. I can check whether I have an ISO copy somewhere, but I really don't want you to spend time on this, because I'm more than happy with SDL and apparently no other users have an issue with the X version.

@pamoroso : you can use Gingko as a drop-in replacement for a regular SDL installation; just move the original ldesdl which is located in maiko/linux.<architecture> somewhere else and instead copy the gingko executable there under the name ldesdl; like this - in case you want to try that at all - the system should behave like the original SDL build. I yet have to add support for CTRL+number keys; what I'm surprised is that Enter doesn't work; it is mapped to Lisp code 76 as in the X version; maybe there is a dependency on num lock (unfortunately I don't have an Enter key on my machine). That you get something different for other symbols if you exchange SHIFT and CTRL is not surprising from my point of view.

[nbriggs]

Yeah, archive.org is offline because they suffered a DDOS attack and were also concerned that their user login data was stolen. Not sure when they'll be back online, but I'll wait for that. Looking at whether it's repeatable on virtual hardware is a background task.

What I'm currently looking at is being able to do a loadup on a system which does not have any X11. Currently "ldeinit" requires X11, since Peter didn't build an SDL version of it, and the SDL code needs a few adjustments to cope in the "INIT" phase. The target consumer for that is (64-bit) Haiku, since they don't have any X, but I'm testing on macOS with SDL2 (while keeping in mind the recently released SDL3 - which, unfortunately, has major API changes - I've still got helping them debug their macOS release on my to-do list).

[rochus-keller]

being able to do a loadup on a system which does not have any X11. Currently "ldeinit" requires X11

Isn't the resulting loadup platform independent? I did a diff with the deployments of different architectures and came to the conclusion that only the maiko executables differ. So you maybe can reuse a loadup from another platform and just replace the executables? I did so with my installation.

[rochus-keller]

@pamoroso : I found a copy/paste error in CTRL key up (FALSE instead of TRUE), but it didn't manifest on my machine (did many times write something and then CTRL+U without failing); so I'm not sure whether that was the cause of your observation. I now also mapped the Enter key to the same Lisp code as Return; would you expect a different behaviour of Enter and Return, or the same? The fixes are commited to Github.

[nbriggs]

A lisp system image, e.g., foo.sysout, is independent of the system you built it on. Building a fresh clean sysout requires both an lde+lde[x|sdl] and an ldeinit. Files that go into ldeinit are compiled with -DINIT, and unfortunately some code in sdl.c depends on a few things like pointers into the Lisp memory that are NOT (and cannot reasonably be) set up on the INIT path prior to SDL code being invoked.

Sure, you can build your sysouts on a different system - but it's unfriendly to make you do that when I can just fix the sdl.c code so you can build a non-X ldeinit.

[nbriggs]

@rochus-keller re the keypad enter -- yes, Return and Enter are completely different, the keypad Enter key was assigned as PAD5, aka the rightmost key of the 5-key Dorado keyset.

[rochus-keller]

@nbriggs : the keypad Enter key was assigned as PAD5, aka the rightmost key of the 5-key Dorado keyset.

I checked in both the original sdl.c and XKeymap.h, and in both cases the Enter key is mapped to Lisp code 76; I implemented it like this in Gingko, but as @pamoroso found out it doesn't seem to work; I can't judge for myself what exactly should happen when you press Enter (I even had to fetch keyboard with an Enter key from the attic since I never use the numpad keys); in the most recent version now I mapped it to the same code as Return.