OLED Re-init on ext_pwr on

mcrosson commented 3 years ago

when ext_pwr comes back online the oled is not re-initialized and stays blank. this is particularly noticeable when coming out of sleep

the below was provided by @Nicell as a hack to work around the problem and suggested the ext_power should be moved to kernel level

#define DISPLAY_INIT_PRIORITY 95

DEVICE_AND_API_INIT(ssd1306, DT_INST_LABEL(0), ssd1306_init,
            &ssd1306_driver, NULL,
            POST_KERNEL, DISPLAY_INIT_PRIORITY,
            &ssd1306_driver_api);

idan commented 3 years ago

Hm, I'd think there is (or ought to be) a separation of display initialization from scene/frame rendering, so display init can be rerun after being powered back on.

I'm looking at this code now as I try to figure out how to use a non-oled display type which probably has a "boot up" sequence for the display. With SPI not I2C, for extra Fun™. Hopefully, I will have something intelligent to say about this soon.

mcrosson commented 3 years ago

This actually needs to be an update to the ssd1306 driver in zephyr proper. There are power management apis that aren't implemented in the upstream driver that need to be implemented.

ext_pwr implements the power management api so updating the upstream driver is the first step to addressing this.

idan commented 3 years ago

Ah got it, thanks for the explanation!

carcinization commented 3 years ago

bumping just because still stymied by this problem :(

Nicell commented 3 years ago

bumping just because still stymied by this problem :(

@carcinization The fix in the description has been implemented, and it mostly fixes the issue. If you're still experiencing it, you may need to increase the external power init-delay-ms for your board.

tokolist commented 3 years ago

@Nicell It seems to be fixed only for turning device on and off completely, but if you turn it off and on, for instance, with RGB_TOG (with CONFIG_ZMK_RGB_UNDERGLOW_EXT_POWER=y), display doesn't get reinitialized. I tested up to 2000 delay which is huge. It looks like display initialization should be added somewhere on external power turning on. Can it be fixed in board configuration or it should be fixed in core?

UPD: Additionally it causes another issue. When you turn device on and external power is off initially, display doesn't get initialized as well even if you turn external power on later. As workaround you can turn external power on and wait about a minute till state gets saved and then turn device off and on so display gets initialized.

Nicell commented 3 years ago

@tokolist This is a known issue. RGB + OLEDs don't play nicely together. The solution presented in this issue only helps for reinitializing after deep sleep/startup. The OLED driver needs to reinitialize itself when moving between power management states that result in power being removed. We also probably need our external power driver to move into and out of the low power devices state when cutting power.

tokolist commented 3 years ago

It seems like there is no good way to reinitialize screen driver in Zephyr, at least I couldn't find one. There is open issue with idea that should allow to reinitialize driver https://github.com/zephyrproject-rtos/zephyr/issues/39896 I was thinking about some workarounds, but they don't look good to me from architectural standpoint.

ddudek commented 2 years ago

I tried some solutions. I managed to trigger ssd1306_init_device method in ssd1306.c after bringing back power, but it fails on first writing on I2C with an error. I even re-tried after adding a significant delay with the same result. Here's the log with some added debug points for time reference:

[00:00:37.083,312] <wrn> zmk: Set ext-power ON now  <--- added
[00:00:37.083,343] <dbg> zmk.zmk_event_manager_handle_from: Listener handled the event
[00:00:37.083,343] <dbg> zmk.zmk_event_manager_handle_from: Listener handled the event
[00:00:37.083,374] <dbg> zmk.split_central_split_run_callback: 
[00:00:37.083,404] <err> zmk: Source not connected
[00:00:37.173,095] <dbg> zmk.kscan_matrix_read: Sending event at 2,0 state off
[00:00:37.173,187] <dbg> zmk.zmk_kscan_process_msgq: Row: 2, col: 0, position: 24, pressed: false
[00:00:37.173,217] <dbg> zmk.position_state_changed_listener: 24 bubble (no undecided hold_tap active)
[00:00:37.173,248] <dbg> zmk.zmk_keymap_apply_position_state: layer: 4 position: 24, binding name: EXTPOWER
[00:00:37.173,278] <dbg> zmk.split_bt_invoke_behavior_payload_delay: 
[00:00:37.173,370] <dbg> zmk.zmk_event_manager_handle_from: Listener handled the event
[00:00:37.173,400] <dbg> zmk.zmk_event_manager_handle_from: Listener handled the event
[00:00:37.173,431] <dbg> zmk.split_central_split_run_callback: 
[00:00:37.583,312] <wrn> zmk: reset oled now   <--- added
[00:00:37.583,343] <wrn> ssd1306: ssd1306 oled re-init now   <--- added
[00:00:37.583,374] <err> i2c_nrfx_twi: Error on I2C line occurred for message 0
[00:00:37.583,374] <err> ssd1306: ssd1306_suspend   <--- added in ssd1306.c in ssd1306_init_device function just before "return -EIO;"
[00:00:37.583,404] <err> ssd1306: Failed to initialize device!

Any ideas how to proceed or is it even a good direction?

ddudek commented 2 years ago

wow, looks like I've made it working by resetting i2c too, I'll try to post working workaround in a few days, if not ping me please

tokolist commented 2 years ago

wow, looks like I've made it working by resetting i2c too, I'll try to post working workaround in a few days, if not ping me please

cool! this is great news! I've already workarounded it in hardware way by connecting OLED directly to battery, but would love to try software workaround, since it should save even more battery life.

Percentnineteen commented 2 years ago

@ddudek - curious to see the workaround and thought I would ping you.

ddudek commented 2 years ago

I wanted to test it a bit more before publishing to avoid confustion, few notes:

I can't seem to fix the issue when the ext_pwr is OFF on startup with loaded settings, so be careful not to save power off, but it works when toggle on -> off -> on
if your device started with ext_pwr OFF just turn it on, wait for settings to be saved, restart
the fix needs sleep 30ms before resetting i2c and 30ms before resetting oled, these are minimums that worked for me
changes only for ssd1306 driver and i2c_nrfx_twi driver (I have nice nano v2), if using other drivers compile will fail
backup everything before trying, I'm not a zephyr developer so it's likely I've made some mistakes

My workaround needed modification of zephyr drivers and their apis, so first (assuming you're in the zmk/app/ folder):

cd ../zephyr
git apply zephyr_oled_i2c_fix.diff.txt

zephyr_oled_i2c_fix.diff.txt

then zmk changes:

cd ../app
git apply app_oled_i2c_fix.diff.txt

app_oled_i2c_fix.diff.txt

I've also made a small changes to auto power-off ext_pwr when OLED goes to sleep in zmk, which drastically improves battery life and makes oled + rgb usable oled_auto_power.diff.txt

I'm new to zephyr and zmk, and also not an experienced C dev, so anyone feel free to check this as first step and maybe find a proper/better solution

Percentnineteen commented 2 years ago

I went ahead and pulled these changes and applied the changes from zephyr* and app*. It seems like it almost works but I do see a bizarre result where the display content is rotated 180 degrees and squashed into what was formerly the "bottom" half of the OLED. As far as I can tell, the oled_auto_power changes shouldn't have any impact on this but I will try to test that at some point as well.

I am running with n!n v1s but I don't think that should matter - the same display driver seems to be used for both version.

I will try to dig in a little bit at some point to see if I can understand what is going on (but I'm a bit out of my depth here, so I can't promise much).

Aside from that issue, though, the core concept seems to work well.

ddudek commented 2 years ago

@Percentnineteen Looks like it was not needed for my screen which is not rotated (I skipped some of the initialization to try shorter sleep) but that might be an easy fix, try this: zephyr_oled_i2c_fix_full_init.diff.txt

the change is in ssd1306.c file, function ssd1306_re_init(...)

Percentnineteen commented 2 years ago

That works. After applying that patch, I can confirm that the functionality works. A huge step forward. I guess the next steps are to try to formalize and shove this into 2 PRs (maybe 3?). I guess one to zephyr itself and at least one to zmk (not sure if you want to split off the "turn off ext_power when display is idle" portion).

EDIT: I forgot - I guess the next thing might be resolving the issue with booting where ext_power is OFF on boot.

Percentnineteen commented 2 years ago

Just one more update in case anyone else is using this - with the oled_auto_off* changes, I had some weird behavior. Sometimes the OLEDs would power off, sometimes they wouldn't come back. Sometimes they would come back with garbage. Getting them back on and working would usually require toggling ext_power and waiting a minute and resetting. This was more of a hassle for me than the potential battery savings so I just backed those changes out.

I might look into it a bit more some time as a gateway into understanding the system but there's no pressure from me. It wasn't something easily reproducible so I'm not even 100% sure that backing out the change fixed it yet...

infused-kim commented 2 years ago

@ddudek thank you so much for creating this fix and sharing it here. It works perfectly for me.

For anyone else, who'd like to try this, but doesn't want to set up a zmk build environment...

You can modify your zmk-config's GitHub Actions build.yml to apply the zephyr patch and then build against a zmk fork that contains the zmk fix.

You can see an example of how to do that in this commit: https://github.com/infused-kim/zmk-config/commit/2617a752739bc3fe6229acdf3e0213e9eb66ead2

And you can find a zmk fork with the patch applied here: https://github.com/infused-kim/zmk/commit/ec0a365cbd3b13ebee3c14c919a786d078a0134e

infused-kim commented 2 years ago

For anyone else interested in this, I figured out there is no need to modify the GitHub build.yml to apply the patch.

I created a fork of zephyr with @ddudek's changes here: https://github.com/infused-kim/zmk-zephyr/commits/v2.5.0%2Bzmk-fixes%2Bkim-fixes

And then I modified my zmk fork's west.yml to use that zephyr fork: https://github.com/infused-kim/zmk/commit/04dfeba4e5788e917314d4d77a0ce7f4344f5d04

Now if you build against that zmk fork you will get a working oled implementation: https://github.com/infused-kim/zmk/tree/my-changes/oled-ext-pwr

You can just reference that repo and branch in your zmk-config's west.yml.

@ddudek are you interested in submitting PRs to the two repos? If not, perhaps I could submit them

bobobo1618 commented 2 years ago

FWIW I took the original patch and updated it for a more modern Zephyr, since it didn't apply cleanly when I tried: https://github.com/bobobo1618/zephyr/tree/fix-oled

Didn't have to make any changes to the ZMK patch I think.

tvollstaedt commented 1 year ago

FWIW I took the original patch and updated it for a more modern Zephyr, since it didn't apply cleanly when I tried: https://github.com/bobobo1618/zephyr/tree/fix-oled

Didn't have to make any changes to the ZMK patch I think.

Hi, I'm desperately trying to apply this patch to my local ZMK setup. I'm using the Zephyr repo you suggested, it seems to contain the Zephyr part of the patch. Applying ddudek's patch to ZMK succeeds, but my build always fails with this error:

/zmk/zmk-firmware/app/src/ext_power_generic.c: In function 'drivers_update_power_state': /zmk/zmk-firmware/app/src/ext_power_generic.c:19:26: error: 'CONFIG_LVGL_DISPLAY_DEV_NAME' undeclared (first use > in this function) 19 | #define ZMK_DISPLAY_NAME CONFIG_LVGL_DISPLAY_DEV_NAME | ^~~~~~~~ /zmk/zmk-firmware/app/src/ext_power_generic.c:66:34: note: in expansion of macro 'ZMK_DISPLAY_NAME' 66 | display = device_get_binding(ZMK_DISPLAY_NAME); | ^~~~

Can someone please point me in the right direction?

bobobo1618 commented 1 year ago

Hi, I'm desperately trying to apply this patch to my local ZMK setup. I'm using the Zephyr repo you suggested, it seems to contain the Zephyr part of the patch. Applying ddudek's patch to ZMK succeeds, but my build always fails with this error:

/zmk/zmk-firmware/app/src/ext_power_generic.c: In function 'drivers_update_power_state': /zmk/zmk-firmware/app/src/ext_power_generic.c:19:26: error: 'CONFIG_LVGL_DISPLAY_DEV_NAME' undeclared (first use > in this function) 19 | #define ZMK_DISPLAY_NAME CONFIG_LVGL_DISPLAY_DEV_NAME | ^~~~~~~~ /zmk/zmk-firmware/app/src/ext_power_generic.c:66:34: note: in expansion of macro 'ZMK_DISPLAY_NAME' 66 | display = device_get_binding(ZMK_DISPLAY_NAME); | ^~~~

Can someone please point me in the right direction?

I just blindly did this: https://github.com/bobobo1618/zmk/blob/8af6847b8883daf52eb9cb9a9867bd8627124624/app/src/ext_power_generic.c#L19

It's almost certainly incorrect but it fixes the compile failures.

I believe it's unnecessary if you have a display enabled though.

t18n commented 1 year ago

infused-kim's folk seems to be outdated. Do we have any development on this, even a hacky one?

t18n commented 1 year ago

It seems like the problem caused by the external power being cut off. So it is a known problem, but for a keyboard newbie like me, it was not obvious how to fix it. Here are the things I did:

Assign &ext_power EP_TOG to your_board.keymap file
Build and flash the keyboard
Click on the EP_TOG button (at this stage, I clicked quite a few times, and the OLED is still dark). Eventually, I accidently found out that I need to click on the bootloader button once for the OLED to turn on.

But yeah, it is turned on now. It is amazed that none of the step-by-step fix were mentioned anywhere. Most of the modification I found was so complex that I don't even know where to put the fix. Hope it helps someone.

t18n commented 1 year ago

It seems like the problem caused by the external power being cut off. So it is a known problem, but for a keyboard newbie like me, it was not obvious how to fix it. Here are the things I did:

Assign &ext_power EP_TOG to your_board.keymap file
Build and flash the keyboard
Click on the EP_TOG button (at this stage, I clicked quite a few times, and the OLED is still dark). Eventually, I accidently found out that I need to click on the bootloader button once for the OLED to turn on.

But yeah, it is turned on now. It is amazed that none of the step-by-step fix were mentioned anywhere. Most of the modification I found was so complex that I don't even know where to put the fix. Hope it helps someone.

Percentnineteen commented 1 year ago

To get the OLED to turn back on, the required steps are:

Turn VCC on (EP_ON is the most reliable)
Wait 60s for the setting to write to flash
Power cycle or reset the board by pressing the reset button once.

On Wed, Aug 9, 2023, 5:46 PM Turbo Ninh @.***> wrote:

It seems like the problem caused by the external power being cut off. So it is a known problem, but for a keyboard newbie like me, it was not obvious how to fix it. Here are the things I did:

Assign &ext_power EP_TOG to your_board.keymap file

Build and flash the keyboard

Click on the EP_TOG button (at this stage, I clicked quite a few times, and the OLED is still dark). Eventually, I accidently found out that I need to click on the bootloader button once for the OLED to turn on.

But yeah, it is turned on now. It is amazed that none of the step-by-step fix were mentioned anywhere. Most of the modification I found was so complex that I don't even know where to put the fix. Hope it helps someone.

— Reply to this email directly, view it on GitHub https://github.com/zmkfirmware/zmk/issues/674#issuecomment-1672278597, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC4HWRGECLS6PJ4KJER6QTDXUQHLLANCNFSM4XLW5Y3A . You are receiving this because you were mentioned.Message ID: @.***>

NellyWhads commented 1 year ago

@t18n @Percentnineteen Are your steps outlined performed in addition to the patches outlined above? Or is this the "easy no-patch-fix" that can be used without the patches?

Percentnineteen commented 1 year ago

Those steps work without the patch.

On Thu, Nov 16, 2023, 8:38 AM NellyWhads @.***> wrote:

@t18n https://github.com/t18n @Percentnineteen https://github.com/Percentnineteen Are your steps outlined performed in addition to the patches outlined above? Or is this the "easy no-patch-fix" that can be used without the patches?

— Reply to this email directly, view it on GitHub https://github.com/zmkfirmware/zmk/issues/674#issuecomment-1814568957, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC4HWRC72WJ3BFIGKDQW2WTYEYQPZAVCNFSM4XLW5Y3KU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRGQ2TMOBZGU3Q . You are receiving this because you were mentioned.Message ID: @.***>

m42e commented 9 months ago

Is someone currently working on this? I have got my new keyboard with an oled and would be willing to dig into it. But if somebody is already on it I would try to support if I can.

petejohanson commented 9 months ago

Is someone currently working on this? I have got my new keyboard with an oled and would be willing to dig into it. But if somebody is already on it I would try to support if I can.

Check out https://github.com/zmkfirmware/zmk/pull/1775

Which includes the SSD1306 driver fixes, as well as a fully integrated power domain setup to make RGB + OLED not fight over VCC power enabling, and allows multiple power domains for hardware designs that include such things.

zmkfirmware / zmk

OLED Re-init on ext_pwr on #674

For anyone else, who'd like to try this, but doesn't want to set up a zmk build environment...