esp32 board cant connect (Fixed)

[incosa]

for windows 10 i installed your app and selected the com port that my board was on. then your app wouldn't connect to the board and said couldn't connect.

I can open putty or thonny and it will connect to the esp board with no problems.

is there something i am missing? i used the exe file from clicking on windows in the git page

[Necromantic]

Same problem here. Wanted to try this out after seeing the Hackaday article. I can't seem to get it to connect. It's the AZ-Delivery Dev Kit.

[jczic]

Hi @incosa, @Necromantic, I think I know the problem, MPY-Jama always connects to the serial port via a boad rate of 115200 bps. This is not necessarily accepted on all ESP32 cards and maybe yours. What did you select in the other tools please? I definitely need to add this in the next version.

[IngmarG]

I confirm this issue. I tried to connect a couple of dozen different ESP32 boards (21 to be exact). None of them connect. It doesn't matter what settings I set in device manager, or whatever. It never connects. Putty e.g. on 115200 connects without problem at 115200.

[jczic]

@IngmarG, what is your firmware on these boards ? The same ? It's not normal but I can understand that with the type of boards and the type/version of firmware. Thank you again.

Other: Do you have a program running on your boards when you connect it?

[IngmarG]

@IngmarG, what is your firmware on these boards ? The same ? It's not normal but I can understand that with the type of boards and the type/version of firmware. Thank you again.

Other: Do you have a program running on your boards when you connect it?

E.g. ESP-32 Dev Kit C V4 by AZ-Delivery, connected to the Arduino IDE (just connecting to the internet via my WiFi and getting the time)

Now, trying to connect to the MPY-Jama IDE (first disconnected of course from Arduino), and powered down.

With result:

and

This happens with ALL devices, no matter what is running on it (Meshtastic, ESPHome, Arduino programs, Tasmota, ...)

[IngmarG]

Am I correct in assuming that MPY-Jama can not flash itself for the first time? Meaning that I have to use esptool.py e.g. to flash the first binary?

Maybe that's the reason it won't connect?

[jczic]

Can you start your ESP32, connect to it directly via putty and press CTRL+A to see if the REPL switches to RAW mode. If it works, can you copy/paste the contents of the REPL here? I think there is a difference in this area from what I have been able to test and I would have to see what it looks like on your end.

Concerning the firmware flash, MPY-Jama uses esptool. So you can flash an image without connect in first.

[jczic]

By the way, you must have a firmware with MicroPython on your ESP32 beforehand because the serial connection is done via the MicroPython REPL, I didn't even think about it at the time. Obviously, on an ESP32 with another type of firmware, it will not work. (MPY = MicroPython)

-> MicroPython port to ESP32 GitHub

[incosa]

Im using MP 1.19.1 i tried with 2 ftdi cables.

[jczic]

@incosa, could you give me a screen shot from a standard serial term at the boot, after CTRL+A pressed please ? My mail is on my github account. Thank you 👍🏻 because I don't understand this problem.

[incosa]

@incosa, could you give me a screen shot from a standard serial term at the boot, after CTRL+A pressed please ? My mail is on my github account. Thank you 👍🏻 because I don't understand this problem.

will do.

FYI i just tried another board i have with 1.18.1 and it worked. ill check our the 1.19.1 board and see what else i can find.

[IngmarG]

By the way, you must have a firmware with MicroPython on your ESP32 beforehand because the serial connection is done via the MicroPython REPL, I didn't even think about it at the time. Obviously, on an ESP32 with another type of firmware, it will not work. (MPY = MicroPython)

-> MicroPython port to ESP32 GitHub

Kind of illogical as e.g. any other "OS" can flash over a pre-existent different one. One just needs to go into boor mode. I can flash any Arduino compiled image over Meshtastic, ESPHome or Tasmota for instance.

I'll give it a try, but where do I find an image precompiled that will work with any of the devices that I have (literally dozens of different brands, like Heltec, Espressif dev kit (from espressif), other dev kits form AZ-delivery, fakes from Aliexpress, ....

BTW: I am a KS backer of MicroPython.

[jczic]

In fact @IngmarG, the software is dedicated to MicroPython functions but mainly for the ESP32 port. It simply uses the installed esptool (if it finds it) to flash a firmware image. In the end, I would like to extend it to as many cards as possible but since it's only the beginning, I couldn't check everything in the different firmwares and the way to communicate with the Raw REPL. I hope to be able to fix all the problems soon :) (KS backer ! Nice ! Do you have an idea for an another issue that I have : https://github.com/micropython/micropython/issues/10718 ?)

[jczic]

@incosa : I've the 1.19.1 with IDF v4.4.1 and it works but indeed, there must be something different for the REPL. The binary version is not up to date but the latest commits also fix the fact that it can interrupt a running program to connect.

[IngmarG]

In fact @IngmarG, the software is dedicated to MicroPython functions but mainly for the ESP32 port. It simply uses the installed esptool (if it finds it) to flash a firmware image. In the end, I would like to extend it to as many cards as possible but since it's only the beginning, I couldn't check everything in the different firmwares and the way to communicate with the Raw REPL. I hope to be able to fix all the problems soon :)

(KS backer ! Nice ! Do you have an idea for an another issue that I have : micropython/micropython#10718 ?)

Managed to flash it (esp32-20220618-v1.19.1 from https://micropython.org/download/esp32/

Now let's see if I can play around a bit.

As to what concerns your other issue: Sorry, I'm an old school programmer: Algol68, ADA, Fortran, Pascal, Delphi, Basic, Forth, Prolog, a lot of assemblers, and my first one was actually hand coding to hex on 1802, 6502, and especially SC/MP (INS8060).

Being retired this is an (obsessive) pass time.

[jczic]

Hey! Excellent @IngmarG :) Me C/C++, ASM, Basic, Pascal, Delphi, .Net, J2EE, Javascript, Python, Web... You know the ASN1 logic ? It' fun.. I loved Delphi at it startup! And I loved .Net (C#) just after.

Coding, this has been over than 30 years old for me... :'D

[rdagger]

I've found on a Windows 10 PC that the first connection attempt to the ESP32 fails but a second attempt usually works. I don't do anything between the attempts. I just try to connect, it fails, and then I hit connect again and it works.

[jczic]

Hi @rdagger, Do you connect it via auto-serial port search or directly by the good serial port? I've Windows 10 but I had not see that... I can retry the first connection 2 times but for auto-search, that can be long. Also, if a program running on the ESP32, the last release (v1.0.3) doesn't halt the current process but the next version do it correctly.

[rdagger]

@jczic I usually specify the actual serial port, but I have tried automatic port detection too. Regardless, the first connection attempt always fails. However, the second attempt usually works. I just tested it both ways with a freshly wiped ESP32.

[jczic]

@rdagger: Either I can reproduce it at home, or if I dont't find the bug, it could be interesting to test, on your side, a modified source file with specific error codes (that I'll add) in order to find where it can come from...

[rdagger]

No problem, I can test whatever you want.

[jczic]

Ok, thank you @rdagger! I'll send you a source file tomorrow

[jczic]

Just @rdagger, when your connection fails, is it instantaneous, after 3 seconds or rather 5 seconds?

[rdagger]

@jczic 3 seconds

[jczic]

Hello @rdagger, @IngmarG, I worked on this problem and maybe the bug is now fixed. I'm not 100% sure because I can't reproduce it but it's a communication problem with the mode change in the REPL. Do you want to try the latest sources (not the latest version) to check if the connection problem is still present?

(Commit https://github.com/jczic/ESP32-MPY-Jama/commit/d46b4aaf0bc5649f0b8608f741c7e367c661d3de)

[rdagger]

@jczic I tried the latest sources and unfortunately the problem still persists. First try fails. Second try works. It only fails on the first try after the ESP32 board is plugged into the USB port. After a successful connection you can exit and restart the program and it will connect on the first try. However, if you disconnect, unplug the ESP32 and then plug it back in then the first connection attempt will fail.

[jczic]

@rdagger, hmm it's very strange, that's works on my Windows 10 and MacOS on many cards. I'm looking but I can't find it. Apparently, it does not manage to detect that the RAW REPL mode has been passed the first time.

[rdagger]

Please take a look at the Pymakr repositories: https://github.com/search?q=org%3Apycom+pymakr I use Pymakr2 in VS Code and its connection works very well in Windows 10.

[IngmarG]

Sorry Jean-Christophe (and others),

I’m currently not following up on this issue. No time.

But please keep me in the loop.

Ingmar

From: rdagger @.> Sent: woensdag 22 februari 2023 17:58 To: jczic/ESP32-MPY-Jama @.> Cc: IngmarG @.>; Mention @.> Subject: Re: [jczic/ESP32-MPY-Jama] esp32 board cant connect (Issue #11)

Please take a look at the Pymakr repositories: https://github.com/search?q=org%3Apycom+pymakr I use Pymakr2 in VS Code https://github.com/pycom/pymakr-vsc and its connection works very well in Windows 10.

— Reply to this email directly, view it on GitHub https://github.com/jczic/ESP32-MPY-Jama/issues/11#issuecomment-1440424831 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACRJHVGAMLJLAKHXMBKAUM3WYZASDANCNFSM6AAAAAAUULBSWI . You are receiving this because you were mentioned. https://github.com/notifications/beacon/ACRJHVGMCCXARPFJ5WEHMBLWYZASDA5CNFSM6AAAAAAUULBSWKWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSV3MRX6.gif Message ID: @. @.> >

[jczic]

Re @rdagger, Could you try again the last update on the repo (not the binaries)? The method of switching between REPL modes has been changed. Thank you very much! 🥇

[rdagger]

@jczic Unfortunately the problem persists.

[rdagger]

I did some debugging. The problem appears to be in the esp32Controller.py init method.
The program made it to the following line before throwing an error: machineNfo = self._exeCodeREPL('import uos; [x.strip() for x in uos.uname().machine.split("with")]', timeoutSec=3) I tried increasing the timoutSec to 7 seconds, but it did not help.

I think the problem might actually be in the preceding line: self._switchToRawMode()

I placed a 2 second sleep before the switchToRawMode command and it fixed the problem. Here's the debugging info:

I think the proper fix would be to determine why the sleep is necessary and then add a wait loop until the required condition is met.

[jczic]

Yes I know it's here, but there's no reason to wait normally... It seems that possibly interrupting a program is not welcome just before switching to RAW REPL. Does it work for you with a sleep(0.030) delay? Because 2 seconds is huge and yet everything is flushed. And moreover, a method "_readUntilEnd" is called before... It's really strange :/

[rdagger]

I tried one second, but it wasn’t long enough, so I switched to two seconds.

[jczic]

@rdagger, I found the problem, it was more vicious than that :) I changed a lot of things and fixed it in the last sources! (a problem of cut in the serial flow, I should have known) It's much cleaner now. Have a great day et merci beaucoup! 👍🏻

[rdagger]

@jczic Unfortunately, it is still not working. The only difference is the error message now reads:

The device on the "COM10" port is not compatible.

[rdagger]

I did some more debugging. I don't know if this helps.

In the method def _readUntil the following line: b = self._repl.read_until(readyBytes)

On the first attemp readBytes equals b'exit\r\n>' but the following is returned: b'ets Jun 8 2016 00:22:57\r\n\r\nrst:0x1 (POWERON_RESET),boot:0x3f (SPI_FAST_FLASH_BOOT)\r\nconfigsip: 0, SPIWP:0xee\r\nclk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00\r\nmode:DIO, clock div:2\r\nload:0x3fff0030,len:4540\r\nho 0 tail 12 room 4\r\nload:0x40078000,len:12788\r\nload:0x40080400,len:4176\r\nentry 0x40080680\r\nMicroPython v1.19.1 on 2022-06-18; ESP32 module (spiram) with ESP32\r\nType "help()" for more information.\r\n>>> '

On the second attempt readBytes still equals b'exit\r\n>' and the following is returned: b'\r\n>>> \r\n>>> \r\nraw REPL; CTRL-B to exit\r\n>'

[jczic]

Oh sorry, i not viewed your second last comment... @rdagger, yes it's normal to receive it, the REPL switch in RAW mode and the b'exit\r\n>' is expected. All is ok now and the expected bytes are not "cut" in the flow. The hard reset method works fine also (there was sometimes a bug before). You receive the boot strings before and the REPL Raw switched mode after by _switchToRawMode. I wouldn't understand because the logic are changed. I created many serial problem and "flow cut" to check it and that's works now. Do you have a timeout in _switchToRawMode? It's very strange... I don't have any problem in many esp32 boards :/

[jczic]

@rdagger: My guess is that you are getting exactly the right bytes but there is a huge timeout between \r\n>>> and the end (exit...). With my latest update you can try to increase the timeout at line 438 to at least 7 seconds to see if it works now.

[rdagger]

The try statement in def _switchToRawMode is throwing an exception when self._readUntil(b'exit\r\n>', timeoutSec=3, lockRead=False) is called, because it doesn't end with the correct bytes. Then in the exception: self._raiseConnectionError() This causes a disconnection which results in a failure.

One work-around is to throw an exception that doesn't disconnect such as: raise ESP32ControllerException('Timeout...')

Then wrap the self._switchToRawMode() in a loop with error catching such as:

for i in range(2):
    try:
        self._switchToRawMode()
        break
    except Exception as e:
        print(f"Switch to REPL failure attempt: {e}")

This works.

[rdagger]

@jczic I did try a 7 second timeout for self._readUntil(b'exit\r\n>', timeoutSec=7, lockRead=False) and it did not work.

[jczic]

Hmm... Thank you @rdagger for your feedback again. Yes, _switchToRawMode raises an exception when it exceeds the timeout. I think that the command to switch to REPL is not written to your ESP32 because it booting. Why it booting just when you connect it?

It can restart after, it can start before and receive commands, but not exactly at the same time as the connection. I think that's the problem because 7 seconds is more than enough time to receive the result of the command. Of course, it never received the command...

[rdagger]

I plugged in the ESP32. I waited 5 minutes. I ran the program, and it still produced the same error.

[jczic]

@rdagger It's crazy :') So it seems that sending CTRL+C reboot your ESP32 when normally only CTRL+D can do it. At startup, my program tries to interrupt any program that might be running. So it sends 2x CTRL+C just in case. Then, it switches to RAW REPL (CTRL+A) mode and waits for the \r\nexit>...

If after all this time it is not received on your ESP32, it is well because this one restarts. If you waited 5 minutes, it had already largely started it is certain. So opening the connection or transmitting 2x CTRL+C makes it restart and the switching to RAW mode is not processed by the chip during this time. If you have a normal terminal, you can try to see what happens by trying CTRL+C on your chip after the start (1 and 2 times). Thank you again!

(another possibility, a program is running on it and it asks for the restart of the chip at the moment of its interruption. Almost impossible but ...)

[rdagger]

I tried rshell but Ctrl-C doesn't seem to do anything except return a new line. I also tried Putty and Pymakr and received the same results. I'm not sure if any of these terminals are actually sending the Ctrl-C because it is also used for copy to clipboard.

Entering REPL. Use Control-X to exit.
>
MicroPython v1.19.1 on 2022-06-18; ESP32 module (spiram) with ESP32
Type "help()" for more information.
>>>
>>> import machine
>>> machine.reset()
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x3f (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4540
ho 0 tail 12 room 4
load:0x40078000,len:12788
load:0x40080400,len:4176
entry 0x40080680
MicroPython v1.19.1 on 2022-06-18; ESP32 module (spiram) with ESP32
Type "help()" for more information.
>>>
>>>
>>>

Is it possible that it isn't rebooting. Instead maybe the serial port is not being flushed?

Ctrl-D performs a soft reboot:

>>>
MPY: soft reboot
MicroPython v1.19.1 on 2022-06-18; ESP32 module (spiram) with ESP32
Type "help()" for more information.
>>>

[jczic]

@rdagger: The sending is well done though, but if it is rebooting, the esp32 does not pass the command to the micropython (the serial port scrolls but the micropython is not launched yet). It doesn't need to clean up because it reads until it actually receives the famous exit... which it never gets. So indeed, by waiting and sending the command a little later, everything works (the 2nd time in your loop as well as after a 2 second delay and also on the second connection). The real question is why the esp32 restarts completely at this point. On a terminal, you can also enter help() to see the different commands and keys accepted by the REPL.

Also, I have never seen a version of REPL that talks about CTRL+X. What version of micropython are you on exactly? --> import platform; platform.platform()

[rdagger]

The CTRL-X is part of rshell. Here are the control commands provided by help: Control commands:

CTRL-A -- on a blank line, enter raw REPL mode CTRL-B -- on a blank line, enter normal REPL mode CTRL-C -- interrupt a running program CTRL-D -- on a blank line, do a soft reset of the board CTRL-E -- on a blank line, enter paste mode

Here is the version:

>>> import platform; platform.platform()
'MicroPython-1.19.1-xtensa-IDFv4.2.2-with-newlib3.0.0'

[jczic]

Ok, I don't see a problem here. And if you comment the line 179 (self.InterruptProgram())?

The question remains "why does it restart at connection time?" .... And it's not a cleanup since the command is never read by micropython.

I could make a hack to keep reading at login time as long as there is content coming in with a special timeout for this case but it's still very strange.

(There used to be holes in the connection but since my update they are fixed. So you have to find out what your problem is, which must be somewhere else too probably).

[jczic]

@rdagger, I made a little hack for your board and others boots problems : (commit : https://github.com/jczic/ESP32-MPY-Jama/commit/1aaf81f47cfaa4b97898fa4071c0310b8088f1ba) That's works?

[rdagger]

@jczic Sorry it doesn't work.

I think the problem is that the ESP32 will not accept any commands until the following message is read:

MicroPython v1.19.1 on 2022-06-18; ESP32 module (spiram) with ESP32
Type "help()" for more information.
>>>

There is a 1-2 second delay before the above message is transmitted.

I used a serial port sniffer to analyze the serial data:

ESP32 Sends-->

rst:0x1 (POWERON_RESET),boot:0x3f (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4540
ho 0 tail 12 room 4
load:0x40078000,len:12788
load:0x40080400,len:4176
entry 0x40080680

Program Sends--> 0x03, 0x03, 0x01 (CTRL-C, CTRL-C, CTRL-A)

ESP32 Sends-->

MicroPython v1.19.1 on 2022-06-18; ESP32 module (spiram) with ESP32
Type "help()" for more information.
>>>

Therefore, the ESP32 never receives the break commands or the raw REPL mode command.