build-esp32 fails with unknown option to `s'

[joysfera]

Apparently the slash in sdk/esp32-esp-idf is processed by sed as the end of the substitute command...

root@e9381f8f5e21:/opt/nodemcu-firmware# bash -x /opt/build-esp32 
+ set -e
+ cd /opt/nodemcu-firmware
+ '[' '!' -f sdkconfig ']'
+ version_file=components/platform/include/user_version.h
+ lua_header_file=components/lua/lua.h
+ export BUILD_DATE COMMIT_ID BRANCH SSL MODULES SDK_VERSION
++ date '+%Y-%m-%d %H:%M'
+ BUILD_DATE='2019-08-27 12:31'
++ git rev-parse HEAD
+ COMMIT_ID=d20778ed0930856e39d4223b1d6ace5dc551d582
++ git rev-parse --abbrev-ref HEAD
++ sed -r 's/[\/\\]+/_/g'
+ BRANCH=dev-esp32
++ grep esp32
++ sed -r 's/.*\((.+)\)/\1/'
++ git submodule status
+ SDK_VERSION='-890a341db429db03a328035b59ca101a5952d096 sdk/esp32-esp-idf'
+ grep -q CONFIG_MBEDTLS_TLS_ENABLED=y sdkconfig
+ SSL=true
+ git diff -b --quiet components/platform/include/user_version.h
+ CAN_MODIFY_VERSION=false
+ '[' -z '' ']'
++ date +%Y%m%d-%H%M
+ IMAGE_NAME=dev-esp32_20190827-1231
+ '[' false = true ']'
+ sed -i 's/\r//g' components/lua/lua.h
+ sed -i '/#define LUA_RELEASE[[:space:]]/ s/$/ " on ESP-IDF -890a341db429db03a328035b59ca101a5952d096 sdk/esp32-esp-idf"/g' components/lua/lua.h
sed: -e expression #1, char 99: unknown option to `s'

[marcelstoer]

SDK_VERSION='-890a341db429db03a328035b59ca101a5952d096 sdk/esp32-esp-idf'

Argh, this is the smoking gun but the real issue is actually one step further away.

https://github.com/marcelstoer/docker-nodemcu-build/blob/de49981b7b2596290837a77e86f5cacfb78b29c6/build-esp32#L22-L23

But see how the sdk/esp32-esp-idf Git submodule currently doesn't provide a version in () anymore:

~/Data/NodeMCU/nodemcu-firmware (dev-esp32) > git submodule status
 3da57e5aa01ebc5c228fd9adfb1f5956c63d1648 components/qrcodegen/qrcodegen (v1.3.1-2-g3da57e5)
 2ee84c8f14adaa8fd1ebfe091c4de348c5474b18 components/u8g2/u8g2 (2.25.10)
 e21641a6c1ddb0e71f7b9e01501fa739786c68b1 components/ucg/ucg (1.5.2)
 890a341db429db03a328035b59ca101a5952d096 sdk/esp32-esp-idf

[joysfera]

Do I understand it correctly that Espressif changed something in their SDK that breaks your build script? Can you please work around that? I've just got an ESP32-POE board from Olimex and was eager to try out Ethernet in NodeMCU :-)

EDIT: I have just understood that originally there was "(v3.2-dev-1239-...)" and it's missing now. Oh well. Can we report it upstream? :)

P.S. I have worked around it in my copy of the script so I am building the image now.

[marcelstoer]

I can't be sure. I first need to figure out how Git produces the output of git submodule status. I suspect that what it puts between () is a tag/release - but only if the revision hash actually corresponds to a tag. If so, then this would mean we're submoduling an untagged ESP-IDF version.

[joysfera]

Hmm, I thought I worked around it but the build failed a moment later with tons of errors so I must have done something incorrect when hardcoding the version number there. I'll rather wait for your fix. Thanks in advance!

[Kempinho]

The Dockerfile is based on Ubuntu 16.04 which installs git version 2.7.4 where git submodule status produces a non-consumable output for the build script:

root@7223251a5ee8:/opt/nodemcu-firmware# git --version
git version 2.7.4
root@7223251a5ee8:/opt/nodemcu-firmware# git submodule status
-3da57e5aa01ebc5c228fd9adfb1f5956c63d1648 components/qrcodegen/qrcodegen
-2ee84c8f14adaa8fd1ebfe091c4de348c5474b18 components/u8g2/u8g2
-e21641a6c1ddb0e71f7b9e01501fa739786c68b1 components/ucg/ucg
-890a341db429db03a328035b59ca101a5952d096 sdk/esp32-esp-idf

on my machine I'm running Ubuntu 18.04 which runs git version 2.17.1 shows the expected (consumable) output:

$ git --version
git version 2.17.1
$ git submodule status
 3da57e5aa01ebc5c228fd9adfb1f5956c63d1648 components/qrcodegen/qrcodegen (v1.3.1-2-g3da57e5)
 2ee84c8f14adaa8fd1ebfe091c4de348c5474b18 components/u8g2/u8g2 (2.25.10)
 e21641a6c1ddb0e71f7b9e01501fa739786c68b1 components/ucg/ucg (1.5.2)
 890a341db429db03a328035b59ca101a5952d096 sdk/esp32-esp-idf (v3.3-beta1-791-g890a341db)

Unfortunately, basing the Dockerfile on Ubuntu 18:04 fixed the issue mentioned here, but my build fails since it is probably using a different version of the toolchain (I did not investigate any further yet)

A workaround for building with the current docker image is hardcoding SDK_VERSION="v3.3-beta1-791-g890a341db" in /opt/build-esp32

[marcelstoer]

I can't confirm that. Once https://github.com/nodemcu/nodemcu-firmware/pull/2898 is merged it works just fine. Even with previous Git versions on Ubuntu 14.04 it produced the expected output.

root@0cd8c0324fac:/tmp/nodemcu# git submodule status
 3da57e5aa01ebc5c228fd9adfb1f5956c63d1648 components/qrcodegen/qrcodegen (v1.3.1-2-g3da57e5)
 2ee84c8f14adaa8fd1ebfe091c4de348c5474b18 components/u8g2/u8g2 (2.25.10)
 e21641a6c1ddb0e71f7b9e01501fa739786c68b1 components/ucg/ucg (1.5.2)
 97e7c14f4e2544ab73aebf372806c17bcdca44bc sdk/esp32-esp-idf (v3.3-rc)
root@0cd8c0324fac:/tmp/nodemcu# git --version
git version 2.7.4

However, one could possibly guard against SDK_VERSION being empty in https://github.com/marcelstoer/docker-nodemcu-build/blob/master/build-esp32#L23

[kit-klein]

I pulled down the branch mentioned in nodemcu/nodemcu-firmware#2898 and updated my submodules. The sdk/esp32-esp-idf version seems to correctly reflect the 'v3.3' tag but the error is still occurring if I attempt to build. I also confirmed I'm on the latest docker image.

kklein@DESKTOP-N7Q073H:/c/_projects/nodemcu-firmware$ git submodule status
 3da57e5aa01ebc5c228fd9adfb1f5956c63d1648 components/qrcodegen/qrcodegen (v1.3.1-2-g3da57e5)
 2ee84c8f14adaa8fd1ebfe091c4de348c5474b18 components/u8g2/u8g2 (2.25.10)
 e21641a6c1ddb0e71f7b9e01501fa739786c68b1 components/ucg/ucg (1.5.2)
 6ccb4cf5b7d1fdddb8c2492f9cbc926abaf230df sdk/esp32-esp-idf (v3.3)
kklein@DESKTOP-N7Q073H:/c/_projects/nodemcu-firmware$ git --version
git version 2.17.1
kklein@DESKTOP-N7Q073H:/c/_projects/nodemcu-firmware$ git submodule status|grep esp32|sed -r 's/.*\((.+)\)/\1/'
v3.3
kklein@DESKTOP-N7Q073H:/c/_projects/nodemcu-firmware$ docker run --rm -ti -v //c/_projects/nodemcu-firmware:/opt/nodemcu-firmware marcelstoer/nodemcu-build build
sed: -e expression #1, char 99: unknown option to `s'
kklein@DESKTOP-N7Q073H:/c/_projects/nodemcu-firmware$ git status
On branch chore/upgrade-idf
Your branch is up to date with 'origin/chore/upgrade-idf'.