Incude SDK and possible the whole toolchain

[con-f-use]

The purpose of this issue is to discuss an e-mail conversation, I had with @cnlohr :

Espressif has sent me a patchwork of various updates to their SDK. I am adding them to https://github.com/cnlohr/esp_nonos_sdk

One big benefit is that it seems with this and the custom linker script, the IRAM issues are a thing of the past. What do you think it would take to make esp82xx also check out the new SDK?

My response in a nutshell:

You mean as in a Make target for cloning/donwloading the SDK [...]? To answer your question, I guess it wouldn't take much to get it running [...]. Submodule git@github.com:cnlohr/esp_nonos_sdk.git into esp82xx [and adapt the make file].

I've thought about [including the whole Toolchain] at one point. Would be an enormous relief for new users. The reason why I didn't, was that it's probably only going to work for a few users that happen to build on a similarly configured OS.

So the SDK part should just be a matter of small makefile changes and a submodule.

For getting and compiling the whole toolchain, there's already my Ubuntu Script that uses pfalcon's Makefile. We could just include that, modify it to use @cnlohr 's patched SDK and call it a day. That would ignore other non-Ubuntu based systems completely.

@cnlohr , you have a better understanding of the toolchain. What exactly is the relationship between the SDK and the rest of the toolchain? Is it just libraries for the esp82xx that are compiled with the xtensa-gcc or is there more?

[con-f-use]

I have forked @pfalcon's esp_open_sdk Makefile and modified it so it uses your version the SDK. Here is my fork: https://github.com/con-f-use/esp-open-sdk

Can @cnlohr test it please? I'm currently traveling and don't have access to an esp or my usual PC. I'm not sure if I got rid of all the patches, that the Makefile usually applies to the SDK. I could also have gotten rid of sensible patches, that the Makefile applies. Since your's seems to be based on SKD version 2.0.0, some of this stuff might actually be sensible:

    echo -e "#undef ESP_SDK_VERSION\n#define ESP_SDK_VERSION 020000" >>$(VENDOR_SDK_DIR)/include/esp_sdk_ver.h
    $(UNZIP) ESP8266_NONOS_SDK_V2.0.0_patch_16_08_09.zip
    mv libmain.a libnet80211.a libpp.a $(VENDOR_SDK_DIR_2.0.0)/lib/
    $(PATCH) -d $(VENDOR_SDK_DIR) -p1 < c_types-c99_sdk_2.patch
    cd $(VENDOR_SDK_DIR)/lib; mkdir -p tmp; cd tmp; $(TOOLCHAIN)/bin/xtensa-lx106-elf-ar x ../libcrypto.a; cd ..; $(TOOLCHAIN)/bin/xtensa-lx106-elf-ar rs libwpa.a tmp/*.o
    $(TOOLCHAIN)/bin/xtensa-lx106-elf-ar r $(VENDOR_SDK_DIR)/lib/libmain.a user_rf_cal_sector_set.o

[cnlohr]

Gaaah. I thought I commented on your issue before. I specifically want to use the repo for the nonos sdk I posted. It is a tweaked version of the 2.0 SDK using files from Espressif, mostly.

I think it would be cool to support the full stack, but, really, I just want a way to easily get users so it automatically checks out that repo to some place like their ~/esp8266 folder or inside the esp82xx folder as a submodule.

The biggest frustration for people seems that we have to let them specify SDK version and that depends on what they're doing. What I've collected is an SDK to rule them all! No copying patches out, tweaking things, etc. I just want to be in control of our own destiny in this regard than at the mercy of esp-open-sdk.

P.S. We are gonna have to come to terms with this git@ thing. If you use the oauth tokens with github, it doesn't have to prompt you for password or anything, either.

[con-f-use]

I understand. What I want to know is, would it be enough to submodule your sdk-version into esp82xx and then set the appropriate path variables in the make files (e.g. SDK and XTLIB in ./esp82xx/common.mf)? I don't fully understand how the SDK integrates into the other tools and whether something form the SDK needs to be compiled client side. Which is not a big deal, I just don't want to spend any time if you can give me straight answer.

Personally, I'm don't get how people are more frustrated with patching and downloading a specific version of the SDK but not with the whole crosstool-NG, bulding gcc for xtensa and getting all the other software for the toolchain.

We are gonna have to come to terms with this git@ thing. If you use the oauth tokens with github, it doesn't have to prompt you for password or anything, either.

Never heard of "oauth tokens" but I agree, that neither HTTPS nor SSH are the solution that fits all. If I had my wish, git would just automatically try everything until something works. I will try to read about it, but if you can give me a quick summary, I wont complain. Guess I should start at git-automation-with-oauth-tokens. Ah wait git config --global credential.helper libsecret seems to use the standard Ubuntu gnome-keyring to get the password. Will test that, it might make HTTPS the way to go for me. It's still silly for git to hardcode authentication methods in .gitmodules.

[con-f-use]

SOLVED

---

Btw. when I use your version of the SDK on the files that ship whit esp82xx, I get:

/home/jan/Desktop/data/tmp/esp-open-sdk/sdk/lib/libmain.a(app_main.o): In function `user_uart_wait_tx_fifo_empty':
(.irom0.text+0x6d8): undefined reference to `user_rf_cal_sector_set'
/home/jan/Desktop/data/tmp/esp-open-sdk/sdk/lib/libmain.a(app_main.o): In function `flash_data_check':
(.irom0.text+0x847): undefined reference to `user_rf_cal_sector_set'

I.e. the firmware you get by

git clone -b dev https://github.com/cnlohr/esp82xx.git
cp esp82xx/Makefile.example Makefile
make project
make all

[cnlohr]

SOLVED

With which project? For most of my projects, I've c&p'd the example espressif code for those functions. I may have missed a few. That feature was added after 1.5.2, I believe.

[con-f-use]

SOLVED

As I said, the example files that currently come with esp82xx. Turns out, it was just a missing empty function of that name. I added it in the dev branch. DISREGARD THIS AND THE LAST TWO POSTS.

[con-f-use]

Only remaining issues:

[...] would it be enough to submodule your sdk-version into esp82xx and then set the appropriate path variables in the make files (e.g. SDK and XTLIB in ./esp82xx/common.mf)? I don't fully understand how the SDK integrates into the other tools and whether something form the SDK needs to be compiled client side.

And where exactly did you run into trouble with the @git thing?

[cnlohr]

For easy use across Windows/Linux/etc. And for integration to the github auth storing thing, I'm really finding HTTPS is better than SSH. Maybe we should include instructions for both whenever working with it?

[con-f-use]

I'm personally using ssh because I didn't need to install stuff, it's more secure and git recommends it. But for the submodules, I'd stick with the HTTPS simply because it's more portable. I will probably write a small note about how to set up git for use with gnome's keyring in Ubuntu. It was not as straight forward as it should have been. Still I'd like to know where you run into trouble, so we can fix that. An inconsistent solution is most annoying.

[cnlohr]

I'm just saying that in contrast to the ssh you seem to prefer. Though, github is suggesting a move away from stored passwords and toward keys.

[con-f-use]

Okay, I think I've included @cnlohr 's version of the sdk now and got a decent mechanism to start new esp82xx projects.

mkdir my_new_esp_project
cd my_new_esp_project
git clone --recursive https://github.com/cnLohr/esp82xx
cp esp82xx/Makefile.example Makefile
make project

# or if you want to push it to a remote git repository instantly:

make gitproject GIT_ORIGIN=https://github.com/YOUR_USER/YOUR_NEW_REPO.git

It's all in the ft-include_sdk branch. Please check that out and test it.

If @cnlohr is happy with it, we can merge into dev, then squash-merge it into master and close this issue.

I wouldn't include the whole toolchain, as we already linked to my download-and-build script and @pfalcon 's esp-open-sdk in the README. That should be enough.

[cnlohr]

Asked elsewhere, just double-checking is the case sensitive?

[con-f-use]

?

[cnlohr]

"CNLohr/esp82XX" but yes. This seems very cool! Please push!

[cnlohr]

This thread has been so drawn out over so long, was it actually done as per your satisfaction?