PyPortal no sync to ESP

[mikeysklar]

Forum issue : PyPortal could be updated with CLI esptool.py (v4.6 or 4.7). Using the 4.8.1 CLI fails as does Web_Serial_ESPtool.

PyPortal ADA# 4116

I was able to reproduce the same behavior as the forum user.

ESP Web Flasher loaded.
Connecting...
Connected successfully.
Try hard reset.
[Object.debug:191] Finished read loop
Error: Couldn't sync to ESP. Try resetting.

[dhalbert]

This works with esptool.py 4.8.1 (testing on a Matrix Portal, but the no_reset is necessary:

$ esptool.py --before no_reset --port /dev/ttyACM1 chip_id
esptool.py v4.8.1
Serial port /dev/ttyACM1
WARNING: Pre-connection option "no_reset" was selected. Connection may fail if the chip is not in bootloader or flasher stub mode.
Connecting......
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting...
Detecting chip type... ESP32
Chip is ESP32-D0WD-V3 (revision v3.0)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 7c:9e:bd:dc:c9:24
Uploading stub...
Running stub...
Stub running...
Warning: ESP32 has no Chip ID. Reading MAC instead.
MAC: 7c:9e:bd:dc:c9:24
Hard resetting via RTS pin...

WebSerial ESPTool does not work, and neither does https://espressif.github.io/esptool-js/. https://esp.huhn.me/ appears to work.

They used to work, I believe, so I'm not sure what's going on here.

[mikeysklar]

The esptool.py 4.8.1 part had been user reported. I did not investigate it as 4.6 / 4.7 were behaving as expected.

The WebSerial ESPTool not working is concerning. At least that part is reproducable for all of us. I'm not familiar with how it works, but I'll investigate further and update here.

[mikeysklar]

@dhalbert - I hooked up a PyPortal again to try the NINA-FW passthrough upgrade. The behavior is the same (as expected).

ESP Web Flasher loaded.
Connecting...
Connected successfully.
Try hard reset.
[Object.debug:191] Finished read loop
Error: Couldn't sync to ESP. Try resetting.
Connecting...
Error: Failed to execute 'open' on 'SerialPort': The port is already open.

I opened up the Chrome --> View --> JavaScript Console or cmd-option-J to see if I could see more about the error and javascript commands being run. More errors just confirming the same "port is already open", but there hyperlinks to the javascript code.

I can see the failure to open is happening in this segment of code:

document.addEventListener("DOMContentLoaded", () => {
  butConnect.addEventListener("click", () => {
    clickConnect().catch(async (e) => {
      console.error(e);
      errorMsg(e.message || e);
      if (espStub) {
        await espStub.disconnect();
      }
      toggleUIConnected(false);
    });
  });

Then it error messages since we couldn't grab the serial port.

function errorMsg(text) {
  logMsg('<span class="error-message">Error:</span> ' + text);
  console.error(text);
}

I'll look into why.

[mikeysklar]

The timeouts I see from esptool.js look like an old issue with the baudrate being out of sync. I know using 115k has been the traditional speed used. Could that have changed in the web interfaces?

esptool.js
Serial port WebSerial VendorID 0x239a ProductID 0x8035
Connecting...Debug: _connect_attempt default_reset false
Debug: Timeout
Debug: Sync
Debug: Sync err Error: Timeout
.Debug: Sync
Debug: Sync err Error: Timeout
.Debug: Sync
Debug: Sync err Error: Timeout
.Debug: Sync
Debug: Sync err Error: Timeout
.Debug: Sync
Debug: Sync err Error: Timeout
.Debug: Sync
Debug: Sync err Error: Timeout
.Debug: Sync
Debug: Sync err Error: Timeout
.Debug: _connect_attempt default_reset true

[dhalbert]

It might be worth trying older versions of Chrome from several months back.

I tried several baud rates myself when trying to get it to work.

[mikeysklar]

Sure, I'll try some older browser releases.

On Wed, Oct 30, 2024 at 6:11 PM Dan Halbert @.***> wrote:

It might be worth trying older versions of Chrome from several months back.

I tried several baud rates myself when trying to get it to work.

— Reply to this email directly, view it on GitHub https://github.com/adafruit/Adafruit_WebSerial_ESPTool/issues/290#issuecomment-2448799497, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPFI5AI5T3QRYTQD6VO3LLZ6F7VFAVCNFSM6AAAAABQGVBEIGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBYG44TSNBZG4 . You are receiving this because you authored the thread.Message ID: @.***>

[mikeysklar]

@dhalbert - Tried three different release of chrome which is a nightmare (btw) to switch versions on. No luck.

I started trying older releases of the the WebSerial_ESPTool and the v1.0.0 March 2023 release worked.

That will make a good starting point for me to compare the code related to serial connect with the newer releases which timeout.

I'll try for a PR over the weekend.

+---------------+--------------------------------+-------------------------------------------------------------------------------------------+
| Date          | Description                    | Output                                                                                     |
+---------------+--------------------------------+-------------------------------------------------------------------------------------------+
| May 2024      | ESP version of esptool.js      | rst:0X1 (POWERON_RESET), boot:0X3 (DOWNLOAD_BOOT (UARTO/UART1/SDIO_REI_REO_V2))            |
|               | not working with PyPortal      | Waiting for downloads                                                                     |
|               | (Issue #141)                   |                                                                                           |
| Oct. 17, 2024 | Adafruit WebESPTool not        | Debug: Sync err Error: Timeout                                                            |
|               | working with PyPortal          |                                                                                           |
|               | (Issue #290)                   |                                                                                           |
|               | Not a permission issue         | sudo chmod a+rw /dev/cu.usbmodem142201                                                    |
|               |                                |                                                                                           |
| May 2024      | MacOS Ventura - Chrome Stable  | ESP Web Flasher loaded. Connecting... Connected successfully. Try hard reset.             |
|               | Version: 130.0.6723.92         | Error: Couldn't sync to ESP. Try resetting.                                               |
| Oct. 2024     | MacOS Ventura - Chrome Beta    | ESP Web Flasher loaded. Connecting... Connected successfully. Try hard reset.             |
|               | Version: 131.0.6778.24         | Error: Couldn't sync to ESP. Try resetting.                                               |
| Oct. 2024     | Ubuntu 24.04.1 LTS             | ESP Web Flasher loaded. Connecting... Connected successfully. Try hard reset.             |
|               | Version: 126.0.6478.114        | Error: Couldn't sync to ESP. Try resetting.                                               |
| Sep 6, 2024   | Adafruit_WebSerial_ESPTool     | ESP Web Flasher loaded. Connecting... Connected successfully. Try hard reset.             |
|               | v1.2.0                         | Error: Couldn't sync to ESP. Try resetting.                                               |
| Aug 26, 2024  | Adafruit_WebSerial_ESPTool     | ESP Web Flasher loaded. Connecting... Connected successfully. Try hard reset.             |
|               | v1.1.0                         | Error: Couldn't sync to ESP. Try resetting.                                               |
| May 2024      | Adafruit_WebSerial_ESPTool     | ESP Web Flasher loaded. Connecting... Connected successfully.                             |
|               | v1.0.0 (Initial Release)       | Chip type ESP32, MAC Address: B4:8A:0A:56:09:B4                                           |
|               |                                | Uploading stub... Running stub... Stub is now running...                                  |
|               |                                | FlashId: 0x16405E, Manufacturer: 5e, Device: 4016, Flash Size: 4MB                        |
|               |                                | Writing data with filesize: 1158144                                                       |
|               |                                | Took 423170ms to write 1158144 bytes                                                      |
|               |                                | To run the new firmware, please reset your device.                                        |
+---------------+--------------------------------+-------------------------------------------------------------------------------------------+

[mikeysklar]

getting closer...watching the files being loaded as WebSerial fires up I found that modifying a single file:

dist/web/index.js

copying the 1.0.0 to the 1.1.0 release allows for a successfully serial connect. Output from swapping the files on a release that had not been able to connect.

ESP Web Flasher loaded.
Connecting...
Connected successfully.
Try hard reset.
Chip type ESP32
Connected to ESP32
MAC Address: B4:8A:0A:56:09:B4
...

[mikeysklar]

@dhalbert - found it.

The hardReset() function is significantly different between the original v1.0.0 WebSerial_ESPTool and later releases.

Both newer releases from Aug/Sep of this year work with a PyPortal by reverting ONLY the hardReset() function back to this original v1.0.0:

dist/web/index.js

    async hardReset(bootloader = false) {
        this.logger.log("Try hard reset.");
        await this.port.setSignals({
            dataTerminalReady: false,
            requestToSend: true,
        });
        await this.port.setSignals({
            dataTerminalReady: bootloader,
            requestToSend: false,
        });
        await new Promise((resolve) => setTimeout(resolve, 1000));
    }

The last two releases that have become incompatible with the ESP32 co-processor boards. They using more elaborate DTR/RTS timing for resets and support more devices.

    async hardReset(bootloader = false) {
        this.logger.log("Try hard reset.");
        if (bootloader) {
            // enter flash mode
            if (this.port.getInfo().usbProductId === USB_JTAG_SERIAL_PID) {
                // esp32c3 esp32s3 etc. build-in USB serial.
                // when connect to computer direct via usb, using following signals
                // to enter flash mode automatically.
                await this.setDTR(false);
                await this.setRTS(false);
                await this.sleep(100);
                await this.setDTR(true);
                await this.setRTS(false);
                await this.sleep(100);
                await this.setRTS(true);
                await this.setDTR(false);
                await this.setRTS(true);
                await this.sleep(100);
                await this.setDTR(false);
                await this.setRTS(false);
            }
            else {
                // otherwise, esp chip should be connected to computer via usb-serial
                // bridge chip like ch340,CP2102 etc.
                // use normal way to enter flash mode.
                await this.setDTR(false);
                await this.setRTS(true);
                await this.sleep(100);
                await this.setDTR(true);
                await this.setRTS(false);
                await this.sleep(50);
                await this.setDTR(false);
            }
        }
        else {
            // just reset
            await this.setRTS(true); // EN->LOW
            await this.sleep(100);
            await this.setRTS(false);
        }
        await new Promise((resolve) => setTimeout(resolve, 1000));
    }

[dhalbert]

Great sleuthing! Is this our code or is it from a third party?

[mikeysklar]

The first short hardReset() snippet above is from the the initial check-in of AdafruitWebSerial ESPTool v1.0.0. It is a little unclear who wrote what as the base code had been passed around a bit. This code works with the ESP32 co-processor boards.

This project was originally written by Melissa LeBlanc-Williams. Nabu Casa ported the code over to TypeScript and in March 2022 took over maintenance from Adafruit. In July 2022, the Nabucasa stopped maintaining the project in favor of an official, but very early release of Espressif's esptool-js. Due to the instability of the tool, Adafruit updated their fork with Nabucasa's changes in November 2022 and took over maintenance once again.

The longer hardReset() above is used in both AdafruitWebSerial ESPTool v1.1.0 and v1.2.0 it is not compatible with the ESP32 co-processor boards.

Since all this code is under Adafruit control I'll make a PR today and try it on most of the ESP32 models. I think it will be a 3 line change.

[dhalbert]

I assume the changes made for hardReset() were done for a deliberate reason. As a workaround, we can back them out, but maybe we should look at the passthrough program or how it is built, to make it work better with the hardReset() changes, if possible. The passthrough sketch is here: https://github.com/adafruit/Adafruit_Learning_System_Guides/blob/main/Adafruit_ESP32_Arduino_Demos/SerialESPPassthrough/SerialESPPassthrough.ino A year or two ago I started compiling with TinyUSB as the base USB stack, to fix a different problem. The current passthrough UF2's were built that way. I will spend a little time on this to see if there's something to be done with handling or ignoring DTR/RTS in the sketch.

[dhalbert]

@mikeysklar I looked at this a little more

If one uses esptool.py with a co-processor board, you don't want to force a reset on that board, etc., so you use --before no_reset, e.g:

esptool.py --port /dev/ttyACM0 --before no_reset --baud 115200 write_flash 0 NINA_W102-1.7.7.bin

See https://github.com/espressif/esptool/blob/88319dbc31b35f569182c68cc4128d2b8075d8f9/esptool/loader.py#L601 for what esptool.py does (not) do when that flag is present.

So does it seem like the new code is trying doing a thorough reset in all cases. Perhaps we should add a checkbox to not reset or add a "Connect without Reset" button?

[mikeysklar]

I like the "no reset" checkbox idea. It would match the CLI usage with esptool.py and that probably is the way to go.

I'll run a few tests on a PyPortal vs ESP32-S3 to see how the reset logic is working. There is a three way conditional that I'm not 100% where each model of ESP32 is landing for the assigned reset procedure. I suspect I can work in a no reset into the existing logic so users do not have to remember to check a box.

[mikeysklar]

Okay, I'm going to with the checkbox. There was no differentiation in the hardReset() logic between a PyPortal and Feather ESP32-S3. Both are treated the same. I'll look into how to add the checkbox.

[mikeysklar]

@dhalbert - I put a little time into adding a No Reset option. It's almost there.

; js/script.js

+---------------------------+-----------------------------------------------+
| Change                    | Description                                   |
+---------------------------+-----------------------------------------------+
| Variable Addition         | const noReset = document.getElementById("noReset"); |
+---------------------------+-----------------------------------------------+
| Event Listener Addition   | noReset.addEventListener("click", clickNoReset);    |
+---------------------------+-----------------------------------------------+
| New Function              | clickNoReset(): Saves noReset state to localStorage |
+---------------------------+-----------------------------------------------+
| loadAllSettings Update    | Loads noReset state from localStorage              |
+---------------------------+-----------------------------------------------+

; index.html

+---------------------------+---------------------------------------------------------------+
| Change                    | Description                                                   |
+---------------------------+---------------------------------------------------------------+
| Added `noReset` Toggle    | Inserted a new toggle next to `darkmode` toggle.              |
|                           | Labeled as "Connect without Reset".                           |
+---------------------------+---------------------------------------------------------------+
| Inline Style for Spacing  | Added `style="margin-right: 15px;"` to `noReset` toggle’s `div`|
|                           | for spacing between `noReset` and `darkmode` toggles.         |
+---------------------------+---------------------------------------------------------------+

I need to add some noReset() support the ESPStub and this should be functional.

[mikeysklar]

Got the ESPStub and it is working well with PyPortal and Feather ESP32-S3 (rev tft).

+-------------------------------+---------------------------------------------------------------------+
| Change                        | Description                                                         |
+-------------------------------+---------------------------------------------------------------------+
| Added `noResetEnabled` Check  | Declared `noResetEnabled` to read the state of the `noReset` toggle.|
+-------------------------------+---------------------------------------------------------------------+
| Conditional Reset Skipping    | Added an `if` statement to skip the reset sequence if `noReset`     |
|                               | is enabled (`noResetEnabled` is `true`).                            |
|                               | Logs a message "No reset requested; skipping hard reset." and       |
|                               | returns early.                                                      |
+-------------------------------+---------------------------------------------------------------------+

[mikeysklar]

adding noReset() button #297

[dhalbert]

Fixed by #297.

[dhalbert]

I updated the pass-through Learning Guide Template to talk about the toggle: E.g. in: https://learn.adafruit.com/upgrading-esp32-firmware/upgrade-all-in-one-esp32-airlift-firmware

[mikeysklar]

Cool. Guide changes look good. Thank you for your assistance.