[FEATURE REQUEST] Interacting with LCD via serial/GCODE

[przemo-c]

When printing remotely we can't interact with prompts on LCD That can block remote print job on crash etc.

We need to have GCODE that sends over serial contents of the LCD and commands for operating the knob and reset button.

[bmuessig]

On my previous job, we worked on exactly this problem on the MK3 in order to create a mostly remote controlled print farm

[bmuessig]

It is possible to hook into the LCD driver and to signal updates to the PC by using the characters 0x80 to 0xff in order to create a second "interleaved" serial stream that carries display data and button presses in the respective direction. Since we had our own custom Go software to communicate with the printer, we could just separate the streams when reading them.

[bmuessig]

It would have surely also worked to silence all existing serial output, but we did not want to do this. As the debug output on the MK3S is always ASCII, the 8th character bit is left unused, which we can re-purpose as flag for our own protocol (a bit similar to how UTF-8 is implemented). If the flag is low, the character is a regular ASCII character and if it is high, the rest of the byte is interpreted differently (e.g. as a draw command from the display, such as seek, print character, etc; or in the other direction as a keypress or scroll action).

[leptun]

https://photos.app.goo.gl/tAMGANgmKMPW1fTDA Just a random thing I did 2 years ago. Obviously not perfect, but printing the display to serial was just a side effect of what I was trying to implement.

[przemo-c]

It wouldn't even have to be realtime. Just upon request as a response to command. Not sure how it's structured but if there's a way to print out what current lcd display should be displaying(redraw screen to serial) as a responce to command and using either the same command with parameter to interact as in simulate knob turn/press and reset.

That would solve issues with crash detection pauses and acknowledging thermal runaway protection and just a troubleshooting reset of the printer.

[bmuessig]

If you need a command to print a full frame and since you can't really read out the LCD, you have to maintain a screen buffer to which to copy any changes that go to the LCD. This buffer could then be printed via G-Code. It would also be possible to send button input via G-Code by injecting forced keypresses and scrolls.

[przemo-c]

If you need a command to print a full frame and since you can't really read out the LCD, you have to maintain a screen buffer to which to copy any changes that go to the LCD. This buffer could then be printed via G-Code. It would also be possible to send button input via G-Code by injecting forced keypresses and scrolls.

That's what I've figured. sort of redraw to serial rather than keep up to date copy or send over serial on each lcd update.

[github-actions[bot]]

This issue has been flagged as stale because it has been open for 60 days with no activity. The issue will be closed in 7 days unless someone removes the "stale" label or adds a comment.

[bmuessig]

Bump as still needed

[github-actions[bot]]

This issue has been flagged as stale because it has been open for 60 days with no activity. The issue will be closed in 7 days unless someone removes the "stale" label or adds a comment.

[bmuessig]

Bump as still needed

[3d-gussner]

This won't be implemented except a volunteer is willing to provide a pull request.

[bmuessig]

I’d be happy to implement it @3d-gussner

[3d-gussner]

Great looking forward to review it. Please be careful with the resources as resource hungry PRs need a lot of good arguments to be considered.

[github-actions[bot]]

This issue has been flagged as stale because it has been open for 60 days with no activity. The issue will be closed in 7 days unless someone removes the "stale" label or adds a comment.

[bmuessig]

I started a new job and didn’t have much time recently. I am still going to implement this feature soon.

[3d-gussner]

@bmuessig Looking forward to see your implementation, thanks for the update.

[Tirpitz93]

Which GCODES are we thinking for this feature? I am thinking for example: M940 to echo the contents of the LCD M941 for sending input events with params E for encoder input with integer increment or decrement. B for button event. N long press. And possibly U, D L, R for direction keys if present or required.

[bmuessig]

@Tirpitz93 That would work. When I started working on the code, I also began implementing basic partial refresh and worked on converting the international characters to ASCII.

I would suggest M941 to send the delta since the last full frame read with M940 and have M942 as an input injection mechanism.

[Tirpitz93]

I would suggest M941 to send the delta since the last full frame read with M940 and have M942 as an input injection mechanism.

My implementation would not track changes to reduce memory footprint. I think the memory footprint is more important than saving a few chars when requesting the LCD contents.

[bmuessig]

@Tirpitz93 We only need one bit per character to store a change flag, which we have free as we’re only using ASCII replacement characters that need 7 bits

[leptun]

Just fyi we use all 8 bits on the lcd because of diacritics. Those are mapped from 0x80 onwards

[Tirpitz93]

My LCD buffer array is (LCD_WIDTH+1) * LCD_HEIGHT chars to give a null terminated string for each line. I guess we could remove the terminator too, and send LCD_WIDTH characters each time. And the previous comment basically addresses the question of unicode characters.

[bmuessig]

There are no Unicode characters in the Prusa firmware. It is using a custom encoding that utilizes the custom characters on the printer‘s LCD controller. My suggestion is to use the ASCII replacement characters for our character buffer. They are already included in the firmware. This gives us one bit to spare as a change flag. We then send the delta by prefixing a conservative run of changed characters by their start index in the buffer. This is easy to do on the fly. The reason I would include a delta mode is that sending 80 bytes periodically would still hog the system more than it needs to be. The serial port slows down the transmission significantly.

[github-actions[bot]]

Thank you for your contribution to our project. This issue has not received any updates for 60 days and may be considered "stale." If this issue is still important to you, please add an update within the next 7 days to keep it open. Administrators can manually reopen the issue if necessary.

[przemo-c]

It's still a needed feature.