esp32 docker configure-esp32 failed

danlu01 commented 5 years ago

Hi, I'm trying to use docker esp32 on windows 10. I followed instructions to Configure modules and features for ESP32, then got following errors with configure-esp32.

I ran: docker run --rm -it -v c:/Users/Daniel/nodemcu-firmware:/opt/nodemcu-firmware marcelstoer/nodemcu-build configure-esp32

got following errors:

Setting IDF_PATH and re-invoking...
make[1]: Entering directory '/opt/nodemcu-firmware'
make[2]: Entering directory '/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig'
make[2]: /opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/check-lxdialog.sh: Command not found
cc -c   -DLOCALE -MMD  /opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/mconf.c -o mconf.o
In file included from /opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/mconf.c:23:0:
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:38:10: error: #include expects "FILENAME" or <FILENAME>
 #include CURSES_LOC
          ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:103:2: error: unknown type name 'chtype'
  chtype atr; /* Color attribute */
  ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:200:16: error: unknown type name 'WINDOW'
 int on_key_esc(WINDOW *win);
                ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:221:17: error: unknown type name 'WINDOW'
 void attr_clear(WINDOW * win, int height, int width, chtype attr);
                 ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:221:54: error: unknown type name 'chtype'
 void attr_clear(WINDOW * win, int height, int width, chtype attr);
                                                      ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:223:21: error: unknown type name 'WINDOW'
 void print_autowrap(WINDOW * win, const char *prompt, int width, int y, int x);
                     ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:224:19: error: unknown type name 'WINDOW'
 void print_button(WINDOW * win, const char *label, int y, int x, int selected);
                   ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:225:18: error: unknown type name 'WINDOW'
 void print_title(WINDOW *dialog, const char *title, int width);
                  ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:226:15: error: unknown type name 'WINDOW'
 void draw_box(WINDOW * win, int y, int x, int height, int width, chtype box,
               ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:226:66: error: unknown type name 'chtype'
 void draw_box(WINDOW * win, int y, int x, int height, int width, chtype box,
                                                                  ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:227:8: error: unknown type name 'chtype'
        chtype border);
        ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:228:18: error: unknown type name 'WINDOW'
 void draw_shadow(WINDOW * win, int y, int x, int height, int width);
                  ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/mconf.c: In function 'conf':
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:97:29: error: 'KEY_MAX' undeclared (first use in this function)
 #define ERRDISPLAYTOOSMALL (KEY_MAX + 1)
                             ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/mconf.c:671:45: note: in expansion of macro 'ERRDISPLAYTOOSMALL'
   if (res == 1 || res == KEY_ESC || res == -ERRDISPLAYTOOSMALL)
                                             ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:97:29: note: each undeclared identifier is reported only once for each function it appears in
 #define ERRDISPLAYTOOSMALL (KEY_MAX + 1)
                             ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/mconf.c:671:45: note: in expansion of macro 'ERRDISPLAYTOOSMALL'
   if (res == 1 || res == KEY_ESC || res == -ERRDISPLAYTOOSMALL)
                                             ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/mconf.c: In function 'show_help':
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/mconf.c:793:27: error: 'stdscr' undeclared (first use in this function)
  help.max_width = getmaxx(stdscr) - 10;
                           ^
In file included from /opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/mconf.c:23:0:
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/mconf.c: In function 'conf_choice':
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/dialog.h:97:29: error: 'KEY_MAX' undeclared (first use in this function)
 #define ERRDISPLAYTOOSMALL (KEY_MAX + 1)
                             ^
/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/mconf.c:855:9: note: in expansion of macro 'ERRDISPLAYTOOSMALL'
   case -ERRDISPLAYTOOSMALL:
         ^
Makefile:171: recipe for target 'mconf.o' failed
make[2]: *** [mconf.o] Error 1
make[2]: Leaving directory '/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig'
make[1]: *** No rule to make target '/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/conf-idf', needed by '/opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/mconf-idf'.  Stop.
make[1]: Leaving directory '/opt/nodemcu-firmware'
Makefile:10: recipe for target 'menuconfig' failed
make: *** [menuconfig] Error 2

>

Can someone tell me what's wrong?

marcelstoer commented 5 years ago

Pity I don't have Windows to verify, I'm on macOS - no clue. @HHHartmann you're on Windows, aren't you? Can I invite you to try to reproduce this?

HHHartmann commented 5 years ago

Sure. I will give it a try this afternoon.

HHHartmann commented 5 years ago

Executing the command directly reveals the problem

root@6bd63424099b:/opt/nodemcu-firmware#  /opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/check-lxdialog.
sh
bash: /opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/check-lxdialog.sh: /bin/bash^M: bad interpreter: No such file or directory

the files have windows style lineendings.

danlu01 commented 5 years ago

What is the work around?

Executing the command directly reveals the problem

root@6bd63424099b:/opt/nodemcu-firmware#  /opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/check-lxdialog.
sh
bash: /opt/nodemcu-firmware/sdk/esp32-esp-idf/tools/kconfig/lxdialog/check-lxdialog.sh: /bin/bash^M: bad interpreter: No such file or directory

the files have windows style lineendings.

marcelstoer commented 5 years ago

Ouch. Users can fix this themselves when they clone the repo. Alternatively we could (maybe should) add a .gitattributes file to the NodeMCU repo. Not sure if

* text=auto
*.sh text eol=lf

would be enough, though.

HHHartmann commented 5 years ago

does it propagate to submodules? The tools directory has a nice template for that. Maybe we can take that for the root path.

tools/.gitattributes

# Enforce Unix newlines
*.css   text eol=lf
*.html  text eol=lf
*.js    text eol=lf
*.json  text eol=lf
*.less  text eol=lf
*.md    text eol=lf
*.svg   text eol=lf
*.yml   text eol=lf
*.py    text eol=lf
*.sh    text eol=lf

danlu01 commented 5 years ago

I've tried line ending fix before cloing: git config core.autocrlf true

Still got the same errors.

Ouch. Users can fix this themselves when they clone the repo. Alternatively we could (maybe should) add a .gitattributes file to the NodeMCU repo. Not sure if
* text=auto
*.sh text eol=lf
would be enough, though.

HHHartmann commented 5 years ago

that would convert lineendings to crlf on checkout which is what already happens.

HHHartmann commented 5 years ago

after experimenting a bit it seems to me that there is no easy way to do this. I can copy a .gitattributes file in sdk\esp32-esp-idf prior to updating the submodule i windows. The it is OK. I also found https://stackoverflow.com/questions/23667416/git-attributes-affecting-submodule which seems to cover the sme case. So it seems that there is no one path way to check out the right thing. So either checking out inside docker or having a two step checkout. Should we change the documentation for checkout in the windows/docker scenario or have another script in docker to prepare-modules which has to be executed once.

HHHartmann commented 5 years ago

@marcelstoer which way should I go?

marcelstoer commented 5 years ago

I'm hesitant...in order to analyze #58 I briefly borrowed a Windows machine today and also looked into this issue.

So either checking out inside docker

This is a no-go for me. Too far away from the regular development workflow.

Should we change the documentation for checkout in the windows/docker scenario

Yes, here and at https://nodemcu.readthedocs.io/en/dev-esp32/en/build/#docker-image

or having a two step checkout...have another script in docker to prepare-modules which has to be executed once.

I was thinking if/how we could automatically handle this correctly. Idea:

let user clone repo incl. submodules as per the documentation
within container: before configure-esp32 detect if the SDK submodule contains files with Windows line breaks, if so
- wipe all submodules
- copy a .gitattributes from within the container (previously packaged into the Docker image) into the submodule folders
- run git submodule update --recursive

HHHartmann commented 5 years ago

@marcelstoer I also noticed that the files checked out under Windows are marked as changed in docker. So reverting the changes inside docker would also fix the line ends. So I would do that prior to each configure-esp32. Can we be safe enough to assume that no developer will ever change a submodule?

So documentation would only need to mention the reverting of changes in the docker documentation, if at all.

marcelstoer commented 5 years ago

So reverting the changes inside docker would also fix the line ends.

True, no need to delete and Git submodule init inside the container.

Can we be safe enough to assume that no developer will ever change a submodule?

I believe so.

So, this would fix the line ending issue for submodules (careful to just revert those!) but not for our own sources. For that we'd still need .gitattributes in our repo.

marcelstoer / docker-nodemcu-build

esp32 docker configure-esp32 failed #57