GT911 does not initialize when M5Paper not connected to USB

The problem

I'm working on making the M5Paper work with ESPHome with some success based on the work from previous attempts. But the last part that does not work is the following:

GT911 fails to initialize when M5Paper does not boot with USB power (on embedded battery, even at 100%).

I have no issue when powered, nor when using M5Stack provided FW.

Which version of ESPHome has the issue?

2024.2.2

What type of installation are you using?

Docker

Which version of Home Assistant has the issue?

2024.2.5

What platform are you using?

ESP32

Board

M5Paper

Component causing the issue

GT911

Example YAML snippet

substitutions:
  device_name: M5Paper

esphome:
  name: ${device_name}
  on_boot:
    - priority: 220.0
      then:
        - it8951e.clear
        - delay: 100ms
        - component.update: m5paper_display
    - priority: -100.0
      then:
        - delay: 10s
        - component.update: m5paper_display

esp32:
  board: m5stack-grey
  framework:
    type: arduino
#    type: esp-idf #Can use either

external_components:
  - source: github://Passific/m5paper_esphome

# Enable logging
logger:
  level: DEBUG

# Enable psram
psram:

spi:
  clk_pin: GPIO14
  mosi_pin: GPIO12
  miso_pin: GPIO13

i2c:
  sda: GPIO21
  scl: GPIO22

display:
  - platform: it8951e
    id: m5paper_display
    cs_pin: GPIO15
    reset_pin: GPIO23
    reset_duration: 100ms
    busy_pin: GPIO27
    rotation: 0
    reversed: False
    update_interval: never

touchscreen:
  - platform: gt911
    display: m5paper_display
    id: gt911_touchscreen
    interrupt_pin: GPIO36

m5paper:
  battery_power_pin: GPIO5
  main_power_pin: GPIO2

Anything in the logs that might be useful for us?

No response

Additional information

Maybe due to powering sequence difference, or powerlevels...

From the original code from M5Stack, touchscreen takes some time to init. (could it linked?)

I tried:
- M5PaperComponent setup order as BUS to set power ASAP, but no luck.
- touchscreen setup_priority to -100 (to let it time to power on), same.
- use esp-idf framework instead of arduino, same again.

esphome / issues

GT911 does not initialize when M5Paper not connected to USB #5591