Waveshare 12in48: fix nim definitions

[suranyami]

It was choking on missing definitions:

/tmp/tmptc7y2ch2/frameos/src/drivers/waveshare/driver.nim(7, 31) Error: undeclared identifier: 'EPD_12in48B_WIDTH'
2024-04-01 14:26:43 candidates (edit distance, scope distance); see '--spellSuggest':
2024-04-01 14:26:43 (2, 2): 'EPD_12in48B_M1_WIDTH'
2024-04-01 14:26:43 (2, 2): 'EPD_12in48B_M2_WIDTH'
2024-04-01 14:26:43 (2, 2): 'EPD_12in48B_S1_WIDTH'

This was fixed by changing the nim definitions to:

  EPD_12in48B_WIDTH* = 1304
  EPD_12in48B_HEIGHT* = 984
  EPD_12in48B_MAX_WIDTH* = 1304
  EPD_12in48B_MAX_HEIGHT* = 984

  EPD_12in48B_M1_WIDTH* = 648
  EPD_12in48B_M1_HEIGHT* = (EPD_12in48B_MAX_HEIGHT div 2)
  EPD_12in48B_S1_WIDTH* = (EPD_12in48B_MAX_WIDTH - 648)
  EPD_12in48B_S1_HEIGHT* = (EPD_12in48B_MAX_HEIGHT div 2)
  EPD_12in48B_M2_WIDTH* = (EPD_12in48B_MAX_WIDTH - 648)
  EPD_12in48B_M2_HEIGHT* = (EPD_12in48B_MAX_HEIGHT div 2)
  EPD_12in48B_S2_WIDTH* = 648
  EPD_12in48B_S2_HEIGHT* = (EPD_12in48B_MAX_HEIGHT div 2)

I also added the missing EPD_12in48.nim which is the other version without Red.

Now it progresses successfully to this point:

2024-04-01 14:40:27 - Copying build folders
2024-04-01 14:40:27 - Applying local modifications
2024-04-01 14:40:27 - Creating build archive
2024-04-01 14:40:27 - No cross compilation. Generating source code for compilation on frame.
2024-04-01 14:40:28 $ cd /tmp/tmpjetybqmy/frameos && nimble assets -y && nim compile --os:linux --cpu:arm64 --compileOnly --genScript --nimcache:/tmp/tmpjetybqmy/build_glkhiyldmapo src/frameos.nim 2>&1
2024-04-01 14:40:28 Executing task assets in /tmp/tmpjetybqmy/frameos/frameos.nimble
2024-04-01 14:40:28 assets/web/index.html ... ok
2024-04-01 14:40:28 assets/web/control.html ... ok
2024-04-01 14:40:28 assets/fonts/Ubuntu-Regular_1.ttf ... ok
2024-04-01 14:40:28 Hint: used config file '/opt/nim/config/nim.cfg' [Conf]
2024-04-01 14:40:28 Hint: used config file '/opt/nim/config/config.nims' [Conf]
2024-04-01 14:40:28 Hint: used config file '/tmp/tmpjetybqmy/frameos/nim.cfg' [Conf]
2024-04-01 14:40:28 Hint: used config file '/tmp/tmpjetybqmy/frameos/config.nims' [Conf]
2024-04-01 14:40:30 ........................................................................................................................................................................................................................................................................................................
2024-04-01 14:40:30 /root/.nimble/pkgs2/jester-0.6.0-a5120cfff525e23be29bb2ff47b6ba2f4b2674a3/jester.nim(1346, 9) Hint: Asynchronous route: myrouter. [User]
2024-04-01 14:40:31 /tmp/tmpjetybqmy/frameos/src/frameos/frameos.nim(1, 36) Warning: imported and not used: 'chroma' [UnusedImport]
2024-04-01 14:40:31 Hint: mm: orc; threads: on; opt: speed; options: -d:release
2024-04-01 14:40:31 128120 lines; 2.436s; 397.246MiB peakmem; proj: /tmp/tmpjetybqmy/frameos/src/frameos.nim; out: /tmp/tmpjetybqmy/build_glkhiyldmapo/frameos.json [SuccessX]

then fails:

The following NEW packages will be installed:
2024-04-01 14:40:33 ntp ntpsec python3-ntp
2024-04-01 14:40:35 0 upgraded, 3 newly installed, 1 to remove and 0 not upgraded.
2024-04-01 14:40:35 Need to get 0 B/446 kB of archives.
2024-04-01 14:40:35 After this operation, 1,291 kB of additional disk space will be used.
2024-04-01 14:40:35 debconf: unable to initialize frontend: Dialog
2024-04-01 14:40:35 debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
2024-04-01 14:40:35 debconf: falling back to frontend: Readline
2024-04-01 14:40:35 debconf: unable to initialize frontend: Readline
2024-04-01 14:40:35 debconf: (This frontend requires a controlling tty.)
2024-04-01 14:40:35 debconf: falling back to frontend: Teletype
2024-04-01 14:40:35 dpkg-preconfigure: unable to re-open stdin:
2024-04-01 14:40:35 dpkg: error: parsing file '/var/lib/dpkg/available' near line 0:
2024-04-01 14:40:35 'utf-8' codec can't decode byte 0xff in position 13: invalid start byte

[suranyami]

Okay, so I tried installing sudo apt install -y ntp ntpsec python3-ntp on the RasPi via ssh manually, and it appears to have been a corrupt file in /var/lib/dpkg/info. The file in question was deleted (sudo rm adwaita-icon-theme.list) and the package removed, then re-installed.

Now I got up to this:

2024-04-01 15:27:35 EPD_12in48b.c: In function ‘EPD_12in48B_Init’:
2024-04-01 15:27:35 EPD_12in48b.c:66:23: error: ‘EPD_M1_CS_PIN’ undeclared (first use in this function); did you mean ‘EPD_CS_PIN’?
2024-04-01 15:27:35 66 | DEV_Digital_Write(EPD_M1_CS_PIN, 1);
2024-04-01 15:27:35 | ^~~~~~~~~~~~~
2024-04-01 15:27:35 | EPD_CS_PIN
2024-04-01 15:27:35 EPD_12in48b.c:66:23: note: each undeclared identifier is reported only once for each function it appears in
2024-04-01 15:27:35 EPD_12in48b.c:67:23: error: ‘EPD_S1_CS_PIN’ undeclared (first use in this function); did you mean ‘EPD_CS_PIN’?
2024-04-01 15:27:35 67 | DEV_Digital_Write(EPD_S1_CS_PIN, 1);
2024-04-01 15:27:35 | ^~~~~~~~~~~~~
2024-04-01 15:27:35 | EPD_CS_PIN
2024-04-01 15:27:35 EPD_12in48b.c:68:23: error: ‘EPD_M2_CS_PIN’ undeclared (first use in this function); did you mean ‘EPD_CS_PIN’?
2024-04-01 15:27:35 68 | DEV_Digital_Write(EPD_M2_CS_PIN, 1);
2024-04-01 15:27:35 | ^~~~~~~~~~~~~
2024-04-01 15:27:35 | EPD_CS_PIN
2024-04-01 15:27:35 compilation terminated due to -fmax-errors=3.
2024-04-01 15:27:35 cp: cannot stat 'EPD_12in48b.o': No such file or directory
2024-04-01 15:27:35 /usr/bin/ld: cannot find EPD_12in48b.o: No such file or directory
2024-04-01 15:27:35 collect2: error: ld returned 1 exit status
2024-04-01 15:27:35 make: *** [Makefile:15: frameos] Error 1
2024-04-01 15:27:35 Command exited with status 2

This seems to be related to the fact that this screen is divided into 4 sections: M1, M2, S1, S2.

[suranyami]

Further progress…

Added definitions for EPD_M1_CS_PIN, etcetera.

Now stuck on:

/tmp/tmp_x0qr9zf/frameos/src/drivers/waveshare/driver.nim(10, 32) Error: undeclared field: 'Unknown'
2024-04-01 16:04:32 /tmp/nimble_34/githubcom_xmonadernimassets_0.2.4_d06724dd7b80fb470542ab932f3a94af78fe2eb1/src/nimassets.nim(3, 8) Warning: use the nimble packages `malebolgia`, `taskpools` or `weave` instead; threadpool is deprecated [Deprecated]
2024-04-01 16:04:32 The command exited with status 1
2024-04-01 16:04:32 Error in src/drivers/waveshare/driver.nim:10:32
2024-04-01 16:04:32 Line 10: let color_option* = ColorOption.Unknown
2024-04-01 16:04:32 ........................................^
2024-04-01 16:04:32 Failed to generate frameos sources

I updated the color definition in VARIANT_COLORS to this:

    "EPD_12in48": "Black",
    "EPD_12in48b": "BlackWhiteRed",

But still getting Unknown color.

[suranyami]

@mariusandra Currently, my biggest challenge is figuring out what kinda regex/string-replacement/template magic is happening that determines this. I've tried looking through the python code, but gotta admit it is hella obfuscated.

[mariusandra]

Hey, regarding the ntp error, I think an apt update before installation would've likely solved it. Not sure though, thanks for reporting.

Regarding the variants, in order to avoid manually instrumenting each frame, I built a system that reads the .c source file of the driver, and figures out its dimensions, what functions it exports, and what details to pass it. For example, whether the Display function takes one or two arguments, and what type of data it expects. We need to convert the image to the right format in FrameOS before sending it on to the C drivers, so these need to match perfectly.

The VARIANT_COLORS object should only be used if autodetection fails. Autodetection works for the "B" driver, as the "red white black" functions always take two arguments and is easy to detect. For the 2-color displays, you sometimes need to specify if it's Black or something else. Running python list_drivers.py will tell you if the widths/heights/colors are properly detected or not. I had fixed it for both drivers in the code that's in main, weird to see it's again broken here?

Will merging with main fix the problem? I had made many changes to waveshare.py to fix the detection that I don't see in your branch.

[suranyami]

Will give that a try now.

[suranyami]

Hmm… getting the same thing:

2024-04-02 09:02:42 Hint: used config file '/tmp/tmpqaqqcapk/frameos/config.nims' [Conf]
2024-04-02 09:02:43 .........................................................................................................................................................................................................................................................................
2024-04-02 09:02:43 /tmp/tmpqaqqcapk/frameos/src/drivers/waveshare/driver.nim(10, 32) Error: undeclared field: 'Unknown'

Just to be clear on the VARIANT_COLORS, it should say:

    "EPD_12in48": "Black",
    "EPD_12in48b": "BlackWhiteRed",

and devices.ts should say:

  { value: 'waveshare.EPD_12in48', label: 'Waveshare 12.48" 1304x984 Black/White' },
  { value: 'waveshare.EPD_12in48b', label: 'Waveshare 12.48" (B) 1304x984 Black/White/Red' },

Correct?

There doesn't seem to be a list_drivers.py file. Where would I find that?

[suranyami]

Ah, found list_devices.py, not list_drivers.py… now, of course, I'm getting ModuleNotFoundError so I'll need to run pip, I guess… ah joy… python packages and all the fun that entails… :-)

[suranyami]

Came to my senses and just ran it in a docker terminal. The output says:

    {"value": "waveshare.EPD_12in48", "label": "Waveshare 12.48\" 1304x984 Black/White/Red"},
    {"value": "waveshare.EPD_12in48b", "label": "Waveshare 12.48\" (B) 1304x984 Unknown"},

[mariusandra]

I gave it a quick attempt, and got past the errors when I set both colors to "Black" under VARIANT_COLORS. Specifying "BlackWhiteRed" for the "B" variant was a mistake, since this was referring to just the first argument of the Display function.

I got it compiling with just changing BlackWhiteRed to Black.

Edit: oops, my bad, it's list_devices.py indeed.

[suranyami]

Okay, there were a number of other missing items, once I'd cleaned up the device & driver definitions: mainly to do with the screen being divided into M1, M2, S1, S2 and a bunch of extra pins.

So, it's fully deploying to the Pi now, without any errors, and the preview in the render panel on the the Frame Controller webpage looks like a nice black/red 2 color dithered picture with the message and a clock.

However… the ePaper display itself hasn't changed.

I noticed a whole stack of differences between the DEV_Config.c and DEV_Config.h files in FrameOS compared to the official driver + test files repo from https://github.com/waveshareteam/12.48inch-e-paper.

Their DEV_Config files were from 2018, but the one you refer to is 2019.

There is also a DEV_GPIOConfig function that is not present in the new DEV_Config file, which is a bit confusing. I'm guessing it's doing the board-specific stuff, which is probably why it's not working yet. I'll add that function into the device driver and try again.

[mariusandra]

Oh wow, that sounds more involved. This then also answers the question of why the 12.48" code isn't included in the main waveshare e-Paper repository. Since this seems like a relatively one-off library, it might be worth just inlining whatever is different from DEV_Config.*, or having a totally separate file altogether.

For further debugging, I'd try to run the 12.48" C examples directly, and see if they update the display. Then work my way backwards to see what's different.

In any case, good luck! I'm off for the night now.

[suranyami]

So, I've tried that but I'm seeing nothing in the FrameController terminal. Any idea where the log files for the service are?

[suranyami]

I got logging coming back from the frame:

metricsload=[0,0.03,0.07] cpuTemperature=47.712 memoryUsage={"total":3976257536,"available":3635470336,"percentage":8.6,"used":252108800,"free":2361454592,"active":595275776} cpuUsage=0

So, it's fully deployed now, but still nothing on the ePaper display. Will go over the DEV_Config stuff with a fine-toothed comb, and inline anything I find missing. Will probably be better to move the stuff I added to the frames files into some EPD_12in48 specific files. Have inlined a fair bit already, but no change yet.

I think it'll just be simpler to use the DEV_Config files from the wave share repo, renamed and referenced in the driver files.

I've already run the C and Python examples and they worked fine. I'm assuming there's nothing specific you've added to the DEV_Config files you added? They were probably just a direct copy, yeah?

Thanks for your help so far! It's pretty exciting to get to this stage!

[suranyami]

Okay, so the problem I have now is that some of the fundamental calls in DevConfig.c for EPD_12in48 are being done in a completely different way to the way they are done in the newer version. For instance, the newer version has this:

void DEV_SPI_WriteByte(uint8_t Value)
{
    lgSpiWrite(SPI_Handle, (char *)&Value, 1);
}

and the older one has this:

void DEV_SPI_WriteByte(UBYTE value)
{
  char i;
  DEV_Delay_us(5);
  switch (software_spi.Mode)
  {
  case Mode0: /* Clock Polarity is 0 and Clock Phase is 0 */
    DEV_Digital_Write(software_spi.SCLK_PIN, 0);
    for (i = 0; i < 8; i++)
    {
      DEV_Digital_Write(software_spi.SCLK_PIN, 0);
      DEV_Delay_us(10);
      if (value & 0x80)
      {
        DEV_Digital_Write(software_spi.MOSI_PIN, 1);
      }
      else
      {
        DEV_Digital_Write(software_spi.MOSI_PIN, 0);
      }
      value = value << 1;
      DEV_Delay_us(10);
      DEV_Digital_Write(software_spi.SCLK_PIN, 1);
      DEV_Delay_us(10);
    }
    break;
  case Mode1: /* Clock Polarity is 0 and Clock Phase is 1 */
    DEV_Digital_Write(software_spi.SCLK_PIN, 0);
    for (i = 0; i < 8; i++)
    {
      DEV_Digital_Write(software_spi.SCLK_PIN, 1);
      if ((value << i) & 0x80)
      {
        DEV_Digital_Write(software_spi.MOSI_PIN, 1);
      }
      else
      {
        DEV_Digital_Write(software_spi.MOSI_PIN, 0);
      }
      DEV_Delay_us(5);
      DEV_Digital_Write(software_spi.SCLK_PIN, 0);
      DEV_Delay_us(5);
    }
    DEV_Digital_Write(software_spi.SCLK_PIN, 0);
    break;
  case Mode2: /* Clock Polarity is 1 and Clock Phase is 0 */
    DEV_Digital_Write(software_spi.SCLK_PIN, 1);
    for (i = 0; i < 8; i++)
    {
      DEV_Digital_Write(software_spi.SCLK_PIN, 1);
      if ((value << i) & 0x80)
      {
        DEV_Digital_Write(software_spi.MOSI_PIN, 1);
      }
      else
      {
        DEV_Digital_Write(software_spi.MOSI_PIN, 0);
      }
      DEV_Delay_us(5);
      DEV_Digital_Write(software_spi.SCLK_PIN, 0);
      DEV_Delay_us(5);
    }
    DEV_Digital_Write(software_spi.SCLK_PIN, 1);
    break;
  case Mode3: /* Clock Polarity is 1 and Clock Phase is 1 */
    DEV_Digital_Write(software_spi.SCLK_PIN, 1);
    for (i = 0; i < 8; i++)
    {
      DEV_Digital_Write(software_spi.SCLK_PIN, 0);
      if ((value << i) & 0x80)
      {
        DEV_Digital_Write(software_spi.MOSI_PIN, 1);
      }
      else
      {
        DEV_Digital_Write(software_spi.MOSI_PIN, 0);
      }
      DEV_Delay_us(5);
      DEV_Digital_Write(software_spi.SCLK_PIN, 1);
      DEV_Delay_us(5);
    }
    DEV_Digital_Write(software_spi.SCLK_PIN, 1);
    break;
  default:
    break;
  }
}

as well as that, there is an entirely different way to initialise the SPI, in the EPD_12in48 DEV_ModuleInit:

  software_spi.SCLK_PIN = EPD_SCK_PIN;
  software_spi.MOSI_PIN = EPD_MOSI_PIN;
  software_spi.Mode = Mode0;
  software_spi.Type = Master;
  software_spi.Clock = 10;

So, at first I tried making a DEV_Config_EPD_12in48.c file, but had the conundrum of now all the rest of the Waveshare python and nim code won't know what to do with it.

Kinda stuck here. My knee-jerk reaction would be to duplicate the whole waveshare dir and call it waveshare_2 or something, and just have a separate DEV_Config.c just for the EPD_12in48. But that seems like overkill.

The problem's exacerbated by the turnaround time… building the docker image takes a good minute or more, and then another couple of minutes to "Save and Re-Deploy". I tried cutting down the docker build time but couldn't figure out how to cache the nimble install step. Most of that time is spent downloading and installing the same nimble packages, which seems a waste.

[mariusandra]

Thanks for all the work with this so far! It's pretty hard for me to provide better guidance since I don't have the display myself, and I'm currently on holiday for another week or so.

I did however just order that display for myself, and will let you know when it arrives and when I get it working. Usually it takes a bit less than a week for shipments to arrive from Waveshare.

[mariusandra]

Hey, finally got my 12.48" frame, and got it to work. The thing that was not obvious to me was that you have to disable SPI in order to get it to work. Their wiki's heading is misleading, though the screenshot is correct:

Redeploying FrameOS would enable SPI, causing the frame not to function. I fixed that, and added an option to explicitly disable SPI for these frames. I also moved all the code into its own folder at drivers/waveshare/epd12in48/. It seems to be working well from my limited testing.