search

MarlinFirmware / Marlin

Marlin is an optimized firmware for RepRap 3D printers based on the Arduino platform. Many commercial 3D printers come with Marlin installed. Check with your vendor if you need source code for your specific machine.
https://marlinfw.org
GNU General Public License v3.0
16.28k stars 19.24k forks source link

Support for TFT and touch screen - need feedback and testing #18129

Closed jmz52 closed 4 years ago

jmz52 commented 4 years ago

TFT screens with optional touch panel can be used to control Marlin Low-level IO uses DMA for background data transfer to TFT screen to minimize impact on Marlin's performance.

Current low-level IO implementation is for HAL STM32 and MCU STM32F1 only. SPI implementation lacks STM32F4 initialization code. FSMC implementation STM32F4 initialization code and (temporary) hardcoded to Zx specific. Implementation for STM32F4 and FSMC on VE chips will follow.

Implemented Marlin_UI class for 320x240 screen. Can be used with or without rotary encoder. Support UBL or advanced pause is not implemented yet.

Tested on MKS Robin v2.4 with TFT v2.0 (ST7789v controller)

Earlier builds were tested on EZT-T1 and SKR Mini v1.1 with 128x128 (st7735 controller) but UI 128x128 became outdated due to changes made during touch support development. UI 128x128 implementation will be uploaded once code is updated.

SPI IO required dedicated hardware SPI interface wich can't be shared with SD card. Should be OK to use same SPI for TFT screen and touch panel.

Build environment for MKS Robin - https://github.com/MarlinFirmware/Configurations/pull/114 https://github.com/jmz52/Configurations/tree/mks_robin_hal_stm32/config/examples/Mks/Robin_HAL_STM32

jmz52 commented 4 years ago

Bootscreen bootscreen

jmz52 commented 4 years ago

Status screen Do not mind grey color of hotends - there are no sensors connected. Hotends, heated bed and chamber will change colors when they are hot. Heated bed and heated chamber will change icons when heater is active.

Hotends, heated bed , heated chamber, fan, feed rate, flow rate are clickable and will lead to corresponding edit screens Settings icon will lead to main menu. SD card will lead to "print from sd" menu when sd is mounted and printer is idle.

status_screen

jmz52 commented 4 years ago

Menu screens With page up, back and page down clickable navigation icons. Click to menu item to select it. Click to selected icon to activate it. Double click logic is used to reduce errors when dealing with smaller screens.

menu menu_with_values

jmz52 commented 4 years ago

Edit screens With decrease, confirm, increase and slider controls. Axis move screen does not have slider due to specific encoder handling.

edit_screen mode_axis

jmz52 commented 4 years ago

Confirm screen With cancel and confirm controls. Selected control is highlighted and another one is dimmed, so you can know with one is selected if you use encoder instead of touch panel. confirm_screen

jmz52 commented 4 years ago

Kill screen kill_screen

jmz52 commented 4 years ago

Related issues https://github.com/MarlinFirmware/Marlin/issues/16787 https://github.com/MarlinFirmware/Marlin/issues/17844 https://github.com/MarlinFirmware/Marlin/issues/16732

jmz52 commented 4 years ago

@hobiseven, would you mind some code adoption and testing on Alfawise Ux0 boards?

hobiseven commented 4 years ago

Gosh... Looks like we do the same thing, different flavors. I have a Little LGVL glue ported and running on Alfawise... See what we are about to test :

https://www.lesimprimantes3d.fr/forum/uploads/monthly_2020_05/IMG_1702.jpg.2878a786f88dc99978ec135913587ea4.jpg

I invite you to read the code I built upon tag 2.5.0.3. This is work in progress. I use extui API.

I send you an invite to my privare repo..

hobiseven commented 4 years ago

@jmz52 https://www.lesimprimantes3d.fr/forum/uploads/monthly_2020_05/IMG_1691.jpg.c92379445e9526c18adb49b1478a7ba2.jpg

LVGL is very nice to use, and the adaptation to get DMAs running in the background is really tiny.

hobiseven commented 4 years ago

@jmz52 something which LVGL allows : dynamic displays : See the small moovie. I get the code from one of the forum member tonight . Please check the small video below :

https://www.lesimprimantes3d.fr/forum/applications/core/interface/file/attachment.php?id=92278

thisiskeithb commented 4 years ago

Did you see MKS’ LVGL port? https://github.com/MarlinFirmware/Marlin/pull/18071

hobiseven commented 4 years ago

NOOOPEEE. So we all on our side do work on the same things!

thisiskeithb commented 4 years ago

Options are nice 🙂

mattdog01 commented 4 years ago

This is beautiful. Absolutely love it. However, are you going to make it work for TFT v1.1? In the current nightly build the "touch button" don't work on my TFT v1.1. They are there but no action. Will this upgrade fix that?

jmz52 commented 4 years ago

@mattdog01 TFT v1.1 needs different calibration data. Check you Configuration.h for code below to see if you are using right numbers. Default are for TVT v2.0

New code should work on TFT v1.1, but you need to change calibration data in https://github.com/jmz52/Marlin/blob/tft_and_touch_screen/Marlin/src/HAL/STM32/tft/xpt2046.h

And make sure you are using build environment from https://github.com/jmz52/Configurations/tree/mks_robin_hal_stm32/config/examples/Mks/Robin_HAL_STM32 Check extra_config.ini for details

//
// ADS7843/XPT2046 ADC Touchscreen such as ILI9341 2.8
//
#define TOUCH_BUTTONS
#if ENABLED(TOUCH_BUTTONS)
  #define BUTTON_DELAY_EDIT  75 // (ms) Button repeat delay for edit screens
  #define BUTTON_DELAY_MENU 100 // (ms) Button repeat delay for menus

  /* MKS Robin TFT v2.0 */
  #define XPT2046_X_CALIBRATION    12013
  #define XPT2046_Y_CALIBRATION    -8711
  #define XPT2046_X_OFFSET           -32
  #define XPT2046_Y_OFFSET           256

  /* MKS Robin TFT v1.1 */
  //#define XPT2046_X_CALIBRATION -11792
  //#define XPT2046_Y_CALIBRATION   8947
  //#define XPT2046_X_OFFSET         342
  //#define XPT2046_Y_OFFSET         -19
#endif
hobiseven commented 4 years ago

@mattdog01 looks like your touch settings are inverted in y. So touch zones are on the opposite side of the screen. We actually have also 3 touch screen flavors on alfawise.

hobiseven commented 4 years ago

@jmz52 I like your print screen. I was more thinking about more arrows for move and less text menus....

MS1987 commented 4 years ago

@jmz52 @hobiseven , can I have your email or please send me email ? I want to develop the touch screen together. My email: lms228@163.com

hobiseven commented 4 years ago

Folks, I probably do not have the expertise and level you guys have... I share my repo with you also @MS1987

I will include a piece of LVGL code ( for the images / screens in the coming days, and push it. The goal for me was to make a POC of what the CPU can handle, without destroying the print performance. I started to build a small state machine , and picket icons googling here and there. @sensei73 asked to copy Alfawise U30 Pro GUI and he started to put together LVGL code to do that.

As lvgl only updates what is nedded, we already do less DMA than with DOGM if I am correct.

looxonline commented 4 years ago

Keen to see this ported to the LP1768/9 HALS. Will open it up substantially. Very nice to see marlin supporting TFT'S natively instead of via gcode as a proxy.

hobiseven commented 4 years ago

Let me ask one of my application manager if he can assign someone to this. I am from NXP.

looxonline commented 4 years ago

Let me ask one of my application manager if he can assign someone to this. I am from NXP.

That would be incredible. If the SPI mode is used it could open the functionality up to many thousands who use the SKR series of motherboards via the EXP2 header.

hobiseven commented 4 years ago

Well, you have a couple of things here : LVGL is hardware independent, and beside the speed , all LVGL code could work on ANY TFT. The right LVGL hal has to be done. and this is far from rocket science. This includes FSMC for ST, SPI for a bunch or other boards, with speed limitations , as well as parallel TFT interfaces, for instance, on LPC1788. Hence, it looks to me that LVGL is the proper vehicule going forward. @thinkyhead do you have any preference on the graphics lib?

hobiseven commented 4 years ago

Bootscreen bootscreen

Hi @jmz52

How did you store that picture in the chip? Is that a jpg decompiled on the fly? or a SD card file? 3202402bytes/RGB565 is 153Kb, Jpeg is about 25Kb... hence my question

mattdog01 commented 4 years ago

Hello @jmz52 I have a functional TFT v1.1 again. Thank you! My Nightly Build downloads didn't have src/HAL/STM32/tft so I added the TFT folder to STM32. Then I was able to update /xpt2046.h with my screens calibration numbers. You had taught me about the calibration difference for v1.1 for the Configuration.h file and it has worked great for a few of the Marlin updates and then it stopped working. I had to use a previously compiled ROBIN.bin to restore printer function.
Prior to your help yesterday I had to revert back to "Marlin-MKS-Robin-ILI9328", that you provided me a while ago, just to be able to change settings on my printer and have it work. So now I am really looking forward to setting up for your awesome GUI.
Did your reply give me enough info to get the new TFT function working on my printer? I tried to follow through it (but it was late in the day) and I had compiling errors that prevented a successful build. I will try again today. Thanks so much for your work on this. It is going to be great to have the TFT experience again but under Marlin. I'm loving Marlin for machine functionality but I have missed the bling of the MKS firmware. Take care.

jmz52 commented 4 years ago

I think some explanation is on order. Originally I've started TFT work for EZT-T1 delta printer. It has mainboard is based on STM32F103RBT6 MCU and 128x128 TFT screen based on st7735 controller. MCU has 128KB flash only meaning Marlin with delta math and HAL_STM32F1 will never fit into it. In order to squeeze firmware into such limited flash I've had to switch to HAL_STM32 and abandon u8glib completely. Fonts and icons were reused, but one does not need an whole library to draw 1bpp images. Instead of expanding existing 12864 graphical interface I've made a new implementation of MarlinUI class dedicated for color screen of particular resolution (128x128). This approach is less flexible that extui, but is has an undeniable benefit of already developed menu system. I am no expert in Marlin's internals, so it would be a tremendous amount of work for me to develop new menu structure. Beside I am not sure how much flexibility is needed for tiny 128x128 screen.

With MarlinUI class implementation, once you have a working status screen you need to implement about 5 drawing functions to get a fully functional menu. It is not be as pretty as UI used on MKS boards or BTT smart screen controllers, but it is way better that x2 upscaled 12864 interface we have on Robin boards for some time now.

Final result was rather satisfying - I was able to fit delta math, SD support, eeprom emulation and code for TFT screen into 128KB. st7735

With new UI working it was a matter of several night to implements new IO subsystem for FSMC and get new UI to MKS Robin TFT. Then I could start working on new display resolution and touch support. MKS_Robin_TFT

I am no familiar with LVGL yet, but if it is lightweight enough to not impact print quality it might be a good foundation for new UI. My only concern here is that it might be too flash consuming for low-end MCUs.

For now I am going to focus on SDIO issues and FSMC support for STM32F103VE and STM32F4 MCUs. This low-level functionality will be needed for either UI implementation.

jmz52 commented 4 years ago

How did you store that picture in the chip? Is that a jpg decompiled on the fly? or a SD card file? 320_240_2bytes/RGB565 is 153Kb, Jpeg is about 25Kb... hence my question

@hobiseven this is raw 16-bit TFT image. There is an external SPI flash on MKS Robin board but I am not using it (yet) https://github.com/jmz52/Marlin/blob/tft_and_touch_screen/Marlin/src/lcd/tft/images/bootscreen_320x240x16.h Other images are in 4 bpp greyscale format colored with dynamically generated palette. 1 bpp and 2 bpp images are supported as well.

sensei73 commented 4 years ago

@jmz52 Interesting, if we have access to the external SPI flash, we could use it to load images as menu. This technique is used on the U30 Pro, less ram consumed.

sensei73 commented 4 years ago

@jmz52 By the way, what is the size of this flash?

hobiseven commented 4 years ago

@jmz52 Indeed we also have a pretty large Spi flash connected on spi2. We could store data there too if needed. So far the basic lvgl lib plus a few menus takes 40kb of prog mem, on the ram side you need preferably at least 2 buffers of 10 full lines> we have for the data+ lvgl buffers a bit more than 50% of chip sram used and we can allocate 24kb to lvgl memory for graphics objects. A basic tile menu such as the one in the above video takes 6kb ram.

Regarding using the spi memory, question is how to load it? We have reused alfawise bootloader and we can load prog mem but I am not sure about spi... we would need either to change alfawise bootloader with one that can load prog mem plus spi mem... or maybe we could check what the current bootloader does with the large longer3d.ui file, as it might store it in the spi...

Do you have a piece of code able to dump an spi mem through serial, or write it into the sdcard??

xamakadesigns commented 4 years ago

Just wondering - any compatibility with MKS TFT32 V4.0? It looks like it is equipped with an STM32F107

I since replaced with Bigtreetech TFT35 running unified firmware. Works much better but still some rough spots before it could fully replace a stock screen. So I have two of the MKS TFT32 that are out of work and looking for a project.

jmz52 commented 4 years ago

@odaruc that depends on MCU used in MKS TFT and TFT screen interface. I've check connection type for BTT TFT screens and here is what I've found:

Do you know if schematics for MKS TFT32 V4.0 is available somewhere?

hobiseven commented 4 years ago

@jmz52 Indeed, you are correct. However, some TFT can also cope with SPI. There is an example with ESP32, a faster CPU, 250Mhz, vs 72 for STM32F103VE, running a TFT with LVGL, through DMA/SPI. might be applicable for some setups. But definitely, a true 16 bit // FSMC is much much faster.

As far as I understood LVGL, the updates are made only on the needed basis, ie, you change an icon, only that icon gets updated, and not the full screen, saving quite some bandwidth.

I still need to integrate the tile example from @sensei73 into my POC. Actually, the LVGL port of MKS is using a similar pages state machine than my embryon state machine I started to build. I requested them to provide me visuals of their LVGL GUI. Did not replied so far.

hobiseven commented 4 years ago

@jmz52 gosh, I realize your issue. You have STM32F103RCT6 MCU and I have STM32F103VET6 ..; We have the luxury of 512Kb flash and 64Kb ram. Indeed, we are right now already over 256Kb...even with only a few tiles. That is a big issue for LVGL on your CPU.

looxonline commented 4 years ago

@jmz52 gosh, I realize your issue. You have STM32F103RCT6 MCU and I have STM32F103VET6 ..; We have the luxury of 512Kb flash and 64Kb ram. Indeed, we are right now already over 256Kb...even with only a few tiles. That is a big issue for LVGL on your CPU.

Actually the STM32F103RCT6 contains 512k flash and there are no fuses blocking it off. I am assuming it just ships untested but in many cases it does work. If you can tell the linker to map non essential elements such as graphics to the upper end of flash then it would be very usable. The 48kB of RAM may be a bit more concerning and I have not seen any reports indicating that the STM32F103RCT6 can take advantage of extra RAM that is mapped but not in the spec sheet.

jmz52 commented 4 years ago

My mistake - EZT-T1 board is based on STM32F103RBT6 (128KB flash, 20KB RAM), not STM32F103RCT6. STM32F103RCT6 is an MCU in SKR Mini v1.1 I've tested code on.

jmz52 commented 4 years ago

About SPI. From there I stand using SPI storage for images (and may be fonts too) is a No2 priority. No1 is to make IO works on SPI and FSMC for F1 and F4 MCUs.

Once the code is works on all boards it will be time to make UI looks pretty - better images, alternative menu structure for TFT screens, STL preview. External SPI flash will be a great help here but I would suggest to make it optional. Most SPI screens have no SPI flash, so it is better to have an option to run with reduced graphics that can fit into progmem.

SPI flash update should be Marlin's responsibility - we can't expect random bootloader to be able to handle Marlin's images. I see two ways to update flash images from SD card - during TFT init phase and via dedicated g-code command.

looxonline commented 4 years ago

About SPI. From there I stand using SPI storage for images (and may be fonts too) is a No2 priority. No1 is to make IO works on SPI and FSMC for F1 and F4 MCUs.

Once the code is works on all boards it will be time to make UI looks pretty - better images, alternative menu structure for TFT screens, STL preview. External SPI flash will be a great help here but I would suggest to make it optional. Most SPI screens have no SPI flash, so it is better to have an option to run with reduced graphics that can fit into progmem.

SPI flash update should be Marlin's responsibility - we can't expect random bootloader to be able to handle Marlin's images. I see two ways to update flash images from SD card - during TFT init phase and via dedicated g-code command.

Absolutely agreed that Marlin should be responsible for the SPI storage update. Requiring bootloaders to do this would make this out of reach for most users as they would not know how to re-flash. Additionally, it would require either manufacturers to add this to their bootloaders or to make their bootloaders open source. Neither of which I feel is going to happen.

hobiseven commented 4 years ago

@jmz52 i actually have done the dma fsmc stuff for f103. I do not have spi screens to test with. And I now looks at possible gui speed impacts although I have dma running in the background which is easier for the cpu than what we had with dogm.

xC0000005 commented 4 years ago

Doesn't the SD FAT library support SPI flash? We could treat the images as files, update becomes a "copy."

vivian-ng commented 4 years ago

@hobiseven Can you share the approach for implementing LVGL? For example, what are the files that can be common across boards, and what are those parts which need to be specifically implemented in the HAL? I am hoping to get something up for ESP32 to use ILI9341 display and using LVGL seems to offer quite some potential for cross-board compatibility with lower-level requirements handled by each HAL.

makerbase-mks commented 4 years ago

@hobiseven I can show serval images about our using of LVGL: nano4 nano3 nano2 nano1

hobiseven commented 4 years ago

@hobiseven Can you share the approach for implementing LVGL? For example, what are the files that can be common across boards, and what are those parts which need to be specifically implemented in the HAL? I am hoping to get something up for ESP32 to use ILI9341 display and using LVGL seems to offer quite some potential for cross-board compatibility with lower-level requirements handled by each HAL.

Hi. All is already written for ESP32 / LVGL. The port is done, although it needs to be integrated within Marlin. 3 main functions to be addressed, well described in LVGL documentation : 1- DMA + Hardware SPI to have a background flush of the display buffers on the frame buffer within IL9341. It has to be DMA and it has to be SPI hardware. 2 - Touchscreen event handler , using a second SPI channel 3 - timer to be added along the step and temp timers

and if you want to debug, a 4th function, to dump messages on the serial line.

hobiseven commented 4 years ago

@hobiseven I can show serval images about our using of LVGL: nano4 nano3 nano2 nano1

Looks good.

What is the final code size? How much RAM did you allocate to LVGL? Have you monitored CPU load?

makerbase-mks commented 4 years ago

@hobiseven ,My MCU has 64K RAM, 512k FLASH. And just now we use about 60% of RAM, 50% of FLASH. For LVGL, we have allocate 20k .

hobiseven commented 4 years ago

@makerbase-mks > Sounds like an stm32f103ZE or VE . I got RAM 50%, including 2x10 line buffers for the DMA, and we can allocate about 24Kb for LVGL RAM,. For flash, as soon as you fit pictures, it goes up, but I am at about 260K. So pretty similar. I make a similar state machine, but way simpler... I will copy yours.

Question is now : When scrolling, have you noticed any speed issue with the printer? There might be the infamous layer shifts... But as we run 32 bit CPUs, we may not bump in the issue.

Well, good work you did! Now, we can all have fun making custom skins for the printers ( themes, or whatever you want to call a custom look).

jmz52 commented 4 years ago

@tpruvot The new build environment for Alfawise U20 - https://github.com/jmz52/Marlin/tree/Alfawise_U20

Compiled firmware can be downloaded from here Please pay extra attention to your heaters - I've implemented support for open-drain gpio output for STM32 HAL, but I have no means to test it on real hardware.

thinkyhead commented 4 years ago

Implemented Marlin_UI class for 320x240 screen. Can be used with or without rotary encoder.

w00t! That saves a lot of work…. I would love to have the standard menu system in settled form on more of these screens, with support for all the usual interface methods:

Of course, no one theme is going to make everyone happy, so we will need to provide a set of standard themes / color schemes.

I was (soon) about to embark on making the menu system fully work on this Ender 3 V2 with a DWIN in 90° rotation and a click-wheel. It would be great to coordinate so that all DWIN / DGUS screens implement the full menu system and accessories in a standardized manner. In general this means having a standard set of names for icons, colors, etc., at the MarlinUI->DWIN level, and setting up defines for the screen size and rotation, the area used for menus versus the status, etc.

ETE-Design commented 4 years ago

@jmz52 Would this Work on the MKS TFT 28 / 32?