Can't get build from tags working

[netskink]

In an effort to be more systematic. Let me document my attempts and results.

Using last verified tag/release

This is the April 19 2021 verified release.

Unless otherwise noted, command execution results are not provided. ie. if no output result, then assume no error messages were generated.

$ cd pycom-micropython-sigfox
# This checks out the revision and creates a branch with same name
$ get checkout tags/v1.20.3.b3 -b v1.20.3.b3

# Supposedly you do this after each time you switch branches.
$ git submodule update --init

$ cd esp32
$ make BOARD=GPY VARIANT=PYGATE clean
$ make BOARD=GPY TARGET=boot VARIANT=PYGATE
Use make SECURE=on [optionally SECURE_KEY ?= secure_boot_signing_key.pem] to enable Secure Boot and Flash Encryption mechanisms.
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
COAP Module Enabled
ESP-NOW Module Enabled
PyGate Enabled
PyEthernet Enabled
Pybytes Enabled
Variant: PYGATE

Incompatible IDF git hash:

d647204 is expected from IDF_HASH from Makefile, but
3394ee5 is what IDF_PATH=/home/davis/progs/notmine/forpycom/pycom-esp-idf is pointing at.

You should probably update one (or multiple) of:
  * IDF_PATH environment variable
  * IDF_HASH variable in esp32/Makefile
  * IDF commit, e.g.
cd $IDF_PATH && git checkout d647204 && git submodule update --init --recursive && git clean -d -f -f && cd -

make: *** [application.mk:1020: CHECK_DEP] Error 1

# attempt to fix the error
$ cd $IDF_PATH && git checkout d647204 && git submodule update --init --recursive && git clean -d -f -f && cd -

$ make BOARD=GPY TARGET=boot VARIANT=PYGATE
Use make SECURE=on [optionally SECURE_KEY ?= secure_boot_signing_key.pem] to enable Secure Boot and Flash Encryption mechanisms.
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
COAP Module Enabled
ESP-NOW Module Enabled
PyGate Enabled
PyEthernet Enabled
Pybytes Enabled
Variant: PYGATE
IDF Version OK! d647204
('xtensa-esp32-elf-gcc version NOT OK. expected:2020r3', 'but I got:', 'xtensa-esp32-elf-gcc (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a) 5.2.0\nCopyright (C) 2015 Free Software Foundation, Inc.\nThis is free software; see the source for copying conditions.  There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\n')
make: *** [application.mk:1023: CHECK_DEP] Error 1

Hmm. perhaps I need to rebuild mpy-cross?

$ cd ..; cd myp-cross; make clean; make; cd ../esp32
$ make clean
$ make BOARD=GPY TARGET=boot VARIANT=PYGATE

Results in same error as before. It complains about cross compiler mismatch. I don't know how to resolve this. 8-(

Using verified tag/release with PYGATE in tag name

This is the Aug 25 2020 verified release. Yes, its a year old.

$ cd pycom-micropython-sigfox
$ git checkout tags/v1.20.2.rc11_pygate -b v1.20.2.rc11_pygate
error: The following untracked working tree files would be overwritten by checkout:
... a lot files are listed
$ git clean -fdx
$ git checkout tags/v1.20.2.rc11_pygate -b v1.20.2.rc11_pygate
error: The following untracked working tree files would be overwritten by checkout:
... a lot files are listed
$ git clean -fd
$ rm -rf lib/nrfx lib/nrf
$ git checkout tags/v1.20.2.rc11_pygate -b v1.20.2.rc11_pygate
$ git submodule update --init
$ cd ..; cd mpy-cross; make clean; make; cd ../esp32
$ make BOARD=GPY VARIANT=PYGATE clean
$ make BOARD=GPY TARGET=boot VARIANT=PYGATE
$ cd $IDF_PATH && git checkout c61fe64 && git submodule sync && git submodule update --init --recursive && cd -
$ make BOARD=GPY TARGET=boot VARIANT=PYGATE
$ make BOARD=GPY TARGET=app VARIANT=PYGATE
Whoot! it builds ok!
$ make VARIANT=PYGATE BOARD=GPY ESPSPEED=115200 ESPPORT=/dev/ttyACM0 flash
Use make SECURE=on [optionally SECURE_KEY ?= secure_boot_signing_key.pem] to enable Secure Boot and Flash Encryption mechanisms.
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
COAP Module Enabled
SIGFOX Module Enabled
Pygate Enabled
PyEthernet Enabled
Pybytes Enabled
Variant: PYGATE
IDF Version OK! c61fe64
checking size of image
1958016 bytes => Size OK
Creating release package in /home/davis/progs/notmine/forpycom/pycom-micropython-sigfox/esp32/build-PYGATE
Release package GPy-1.20.2.rc11.tar.gz created successfully.
checking size of image
1958016 bytes => Size OK
Flashing project
python ./tools/fw_updater/updater.py --port /dev/ttyACM0 --speed 921600 flash -t build-PYGATE/GPy-1.20.2.rc11.tar.gz
Running in FTDI mode
Exception: [Errno 71] Protocol error
make: *** [application.mk:852: flash] Error 2

Hmm. It fails to flash though. 8-(

[netskink]

I noticed that the make command did not pass the baudrate to the actual flash command. See below:

$ make VARIANT=PYGATE BOARD=GPY ESPSPEED=115200 ESPPORT=/dev/ttyACM0 flash
...
.... stuff snipped
....
Flashing project
python ./tools/fw_updater/updater.py --port /dev/ttyACM0 --speed 921600 flash -t build-PYGATE/GPy-1.20.2.rc11.tar.gz
Running in FTDI mode
Exception: [Errno 71] Protocol error
make: *** [application.mk:852: flash] Error 2

I thought that perhaps Ihave the ESPSPEED setting wrong. I could simply try to do the flash command manually with correct baudrate. However, that fails with same error, so its not a baudrate error. Its something in the build method. See below:

$ python ./tools/fw_updater/updater.py --port /dev/ttyACM0 --speed 115200 flash -t build-PYGATE/GPy-1.20.2.rc11.tar.gz
Running in FTDI mode
Exception: [Errno 71] Protocol error

[netskink]

hmm. adjusting down to 9600 and it works.

[peter-pycom]

Supposedly you do this after each time you switch branches. $ git submodule update --init

should be recursive! In addition, I usually don't even bother going through the git hassle of switching between major versions. I have separate folders for firmwares based on idf 3.3.1 (1.20.2.x) and idf 4.1 (1.20.3.x)

('xtensa-esp32-elf-gcc version NOT OK. expected:2020r3', 'but I got:', 'xtensa-esp32-elf-gcc (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a) 5.2.0\nCopyright (C) 2015 Free Software Foundation, Inc.\nThis is free software; see the source for copying conditions. There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\n') make: *** [application.mk:1023: CHECK_DEP] Error 1

this means you're using the wrong xtensa compiler

Running in FTDI mode

I think this is because you're using an old branch, if you take latest pygate fw it should detect pic mode and flash ok with faster speeds. I'm still surprise that you have to go as low as 9600, but well, old code. If you have trouble with latest version, please do let me know.

[netskink]

Hello peter-pycom, many thanks for talking with me about the code. I appreciate it. I truly do.

What is the recursive option you mention? I'm just following along with the readme's suggestion. Personally I tend to avoid git submodules so I am barely aware of how to use it. I believe the guide says anytime you switch branches you need to run that command.

Regarding the FTDI/PIC mode, I did something... I need to redo so I can fully document the process, but I eventually got this working in PIC mode and higher speed:

Running in PIC mode
Script Version: 2.1
Flash operation successful.  

As far as the compiler, I'm simply doing whats available based upon the readme.

All of those are fluff questions though. The real question is what is the latest/best version of softare to use for a pygate/gpy combo to behave as a LTE/LoRaWAN gateway or WiFi/LoRaWAN gateway? Currently I'm using tag v1.20.2.rc11_pygate. Do you suggest a better version?

So, why build a custom version anyway? Well, I had a problem where the TTN downlink messages were being sent to the pygate/gpy combo, but the messages were not being sent to the end lora device. I read a forum post by Peter Deng (If I remember the name correctly) mentioning a patch regarding a fix for this problem. I implemented his patch, but it broke lora altogether. Perhaps because at that time, I did not know the VARIANT=PYGATE switch. With that said, I went back and tried to rebuild as-is, so I could at least get a working pygate/gpy with the uplink working. I figured if that worked, I could then reapply his patch to get downlinks working end to end. However, upon testing with the stock build from the tag it seems that the problem is resolved. My only concern is that this tag is a year old.

Thanks again for your response, Peter,

John

[peter-pycom]

What is the recursive option you mention?

git submodule update --init --recursive

Do you suggest a better version?

latest stable release is: 1.20.2.r4 https://forum.pycom.io/topic/6740/firmware-release-v1-20-2-r4

[netskink]

What is the recursive option you mention?

git submodule update --init --recursive

Cool. thanks. I'll update my notes.

Do you suggest a better version?

Suggest a better version? Me? No. I've just started with the pygate.

latest stable release is: 1.20.2.r4

Ok. Cool. I'll try that one and report back the results.

https://forum.pycom.io/topic/6740/firmware-release-v1-20-2-r4

I'll take a look at that forum post as well.

Many thanks again! I appreciate your expertise.