Create remote serial/flashing dongle

[Natomac]

A specialized flashing dongle that helps us adopt new devices quickly without using USB serial.

A dongle with screen with to pre-flash multiple devices. for adopt. Over the air remote dongle adoption. Some kind of Remote serial port. (solving the MAC OS, and Windows port issues)

Setting this up just on a windows PC with python and the serial port should be easy - estimate [ulno]: 4h adding a remote dongle design of a device to carry around: 1month

[ulno]

This can or should also include or based on the platformio remote serial or esptool remote serial flasher here:

• https://github.com/espressif/esptool/blob/master/esp_rfc2217_server.py
• https://docs.platformio.org/en/latest/core/userguide/remote/cmd_device.html

[mbz4]

Great example here: https://www.matbra.com/2020/02/05/use-remote-serial-port-to-flash-esp.html

Need to test but relies on 'ser2net':

This is ser2net, a program for allowing connections between gensio accepters and gensio connectors. Generally, this would be a network connections to serial ports or IPMI Serial Over Lan (SOL) connections, but there are lots of gensios and lots of options. See gensio(5) for information on gensios.

https://github.com/cminyard/ser2net

to install:

sudo apt-get install ser2net

Then expose a tunnel:

ssh -L 8086:localhost:8086 pi@PI_ADDRESS '/usr/sbin/ser2net -d -C "8086:raw:600:/dev/ttyAMA0:115200"'

Finally:

esptool.py -p socket://localhost:8086 write_flash -fm dio 0x000000 BasicOTA.ino.generic.bin

[mbz4]

Also, esptool.py supports remote flashing of ESP32 devices using the RFC2217 protocol and this allows for communication with networked serial ports, enabling you to connect to an ESP32 over the network as if it were connected locally.

GPT instructions:

Setting Up RFC2217 for Remote Flashing

1. Install the necessary tools: Ensure esptool.py and pyserial are installed on both the client (WSL2 instance) and the server (Windows host) systems.

   pip install esptool pyserial

2. Run the RFC2217 server on the Windows host: Create a script named esp_rfc2217_server.py or use the example provided in pyserial. Here’s how to start the server:

   python -m serial.tools.rfc2217_server -p 4000 COM3

   This command exposes the serial port COM3 over TCP port 4000.

3. Flash the ESP32 from WSL2: From your WSL2 instance, you can now flash the ESP32 using the RFC2217 protocol. Use the following command:

   esptool.py --port rfc2217://<Windows_IP>:4000 --baud 460800 write_flash -z 0x1000 firmware.bin

   Replace <Windows_IP> with the IP address of your Windows host and firmware.bin with the path to your firmware file.

Additional Considerations

• Network Latency: There may be issues with network latency, which can affect the flashing process. If you encounter errors, try reducing the baud rate to improve reliability:

  esptool.py --port rfc2217://<Windows_IP>:4000 --baud 115200 write_flash -z 0x1000 firmware.bin

• Manual Reset: Some setups may require manual resetting of the ESP32 into bootloader mode due to network delays affecting automatic reset signals.

• Using Raw Sockets: If RFC2217 is not supported, you can use raw TCP sockets. This method does not support automatic baud rate changes or reset signals, requiring manual intervention.

For more details and troubleshooting tips, refer to the official esptool documentation and the related GitHub issues

[mbz4]

Tasmota features the JS webserial solution where you can flash devices from a browser tab: https://tasmota.github.io/install/

[mbz4]

@ulno maybe we just host a page w/ this API similar to tasmota and others: https://developer.mozilla.org/en-US/docs/Web/API/Web_Serial_API

[ulno]

Just don't like being forced to use Chrome. I think the little Python serial daemon nicely packaged for different platforms would be my preference to go, especially as we potentially even port it to micropython.

On Thu, May 23, 2024, 22:30 mbz @.***> wrote:

@ulno https://github.com/ulno maybe we just host a page w/ this API similar to tasmota and others: https://developer.mozilla.org/en-US/docs/Web/API/Web_Serial_API

— Reply to this email directly, view it on GitHub https://github.com/iotempire/iotempower/issues/45#issuecomment-2127881637, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAARV4QJQIJPJCCO2QMTWQDZDY7VHAVCNFSM6AAAAAAWWPDG2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRXHA4DCNRTG4 . You are receiving this because you were mentioned.Message ID: @.***>

[mbz4]

From convo w/ @ulno:

• package a custom cross-platform installer script
  • can refer to: https://realpython.com/pyinstaller-python/
  • alternative: https://github.com/Nuitka/Nuitka
• add GUI (tk, wx, simplegui...) to streamline port selection, flashing routine, monitor status, seamless debug/troubleshoot, keep track of connections (multipoint?), logs? etc

need to chunk down the dev process here to individual steps:

1. [ ] prepare working proof of concept demo in vanilla Python
2. [ ] when stable, add GUI
3. [ ] validate working, testing

[mbz4]

• thinking to include GUI in existing sphinx dash makes most sense as a custom extension

• can also add other extensions, like graphviz: https://www.sphinx-doc.org/en/master/usage/extensions/index.html can visualise the network + add some basic stats (maybe?)

• • but taking the static site generator and customising with dynamic elements might get costly; mentioned in https://github.com/iotempire/iotempower/issues/76

• the daemon should only be installed (downloaded binary executable from dashboard) in case the dash detects a webserial/webusb api unsupported browser on load...

• funcionally then the headless daemon forwards everything to

  • the dashboard/landing page w/ a js plugin managing the comms or
  • to caddy (?) or separate site (flask hosted?) and linked on sphinx dash ... so everything gets configured from one place only

i'd rather have a single page open and at a glance see network stats + graphics, access to tools (node-red, flashing, cloud commander etc) and the current docs

once a proof of concept backend gets a bit stable we can investigate this further

[ulno]

Maybe also strongly supporting the idea of looking stronger in textual that should be able to be run in the web - however, let's tread very careful as we are talking here about something that runs in an environment where IoTempwoer might usually not be able to run: windows and not wsl

On Mon, May 27, 2024 at 5:20 PM mbz @.***> wrote:

-

thinking to include GUI in existing sphinx dash makes most sense as a custom extension

can also add other extensions, like graphviz: https://www.sphinx-doc.org/en/master/usage/extensions/index.html can visualise the network + add some basic stats (maybe?)

  - but taking the static site generator and customising with dynamic
  elements might get costly; mentioned in #76
  <https://github.com/iotempire/iotempower/issues/76>

-

headless daemon forwards everything to

• the dashboard/landing page w/ a js plugin managing the comms or
  • to caddy (?) or separate site (flask hosted?) and linked on sphinx dash

i'd rather have a single page open and at a glance see network stats + graphics, access to tools (node-red, flashing, cloud commander etc) and the current docs

once a proof of concept backend gets a bit stable we can investigate this further

— Reply to this email directly, view it on GitHub https://github.com/iotempire/iotempower/issues/45#issuecomment-2133584629, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAARV4XRFFH2HOP65K4ALALZEM6MDAVCNFSM6AAAAAAWWPDG2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZTGU4DINRSHE . You are receiving this because you were mentioned.Message ID: @.***>

[ulno]

After several tries, I managed to run the rfc2217 server on Windows and flash slowly with the following command an esp32 (m5stickc_plus):

esptool.py --port rfc2217://192.168.25.69:4000?ign_set_control --baud 115000 write_flash -z 0x1000 firmware.bin