search

espressif / crosstool-NG

crosstool-NG with support for Xtensa
Other
117 stars 62 forks source link

Please upload esp-12.2.0 to platformio registry #41

Closed Jason2866 closed 1 year ago

Jason2866 commented 1 year ago

@antmak to make it possible for developers to adopt Platformio projects to GCC 12.2 Personally i would like to test/support the started Arduino port (based on IDF5.1) with Platformio. Building the framework with Arduino Lib Builder does work. Sadly it cant be used with Platformio since the needed gcc 12.2 toolchains are not available from Platformio registry. (fyi @me-no-dev)

Jason2866 commented 1 year ago

ping @antmak @igrr
I would like to support development of the Arduino port based on IDF 5.1 with the IDE Platformio. Currently it is not possible since the needed 12.2.0 toolchains are not in Platformio registry.

me-no-dev commented 1 year ago

@Jason2866 this needs to be synced with PIO people, because usually it uses the latest toolchain, which will not work with 4.x.

p.s. @valeros @ivankravets

Jason2866 commented 1 year ago

@me-no-dev No, uploading to Platformio registry does not break anything or needs adoption in any code. To use the (new) toolchains from registry the user has to change platformio settings (which is easy). One of the nice features of Platformio. The standard used tools are defined here https://github.com/platformio/platform-espressif32/blob/develop/platform.json

me-no-dev commented 1 year ago

as far as I remember, this was not the case. Let's wait for confirmation and I will upload the toolchain if it will not cause issues (would not be selected and used by folks using ESP-IDF v4.4 or Arduino 2.0.x)

ivankravets commented 1 year ago

@me-no-dev, yes, you can publish the new version of the toolchain. It will not affect existing users.

See the list of dependencies for the latest dev-platform version https://github.com/platformio/platform-espressif32/blob/develop/platform.json#L35

me-no-dev commented 1 year ago

@Jason2866 done

Jason2866 commented 1 year ago

@me-no-dev Great, but having problems to install via Platformio. The version scheme used seems to be not the usual Platformio scheme. Can you post the package.json you used?

ivankravets commented 1 year ago

@Jason2866 , thanks for the report. We've just added more strict version validation on our backend infrastructure.

@me-no-dev , could you re-upload all packages and update their version field to conform SemVer specifications? https://semver.org/

If you want to use "date" as a patch component, it will look like 12.2.0+20230208.

P.S.: We will remove previously published packages with broken versions.

me-no-dev commented 1 year ago

Sure! Rebuilding now

me-no-dev commented 1 year ago

@ivankravets any idea what happened here: https://github.com/espressif/arduino-esp32/actions/runs/4742047281/jobs/8420309733?pr=8099#step:4:659

ivankravets commented 1 year ago

@me-no-dev known issue, sorry. @valeros is working on the fix and will come here with updates. You will just need to restart the CI task.

me-no-dev commented 1 year ago

@ivankravets maybe good idea to poke here about going through some of the changes in Arduino v3.0.0 so that we can enable PIO CI again? And that you are aware of what is coming? The main difference is that we now have the SDK as external package installed by Arduino IDE. You have two ways to approach this:

@Jason2866 you aware of anything else breaking?

Jason2866 commented 1 year ago

@me-no-dev The mentioned points are the ones i have encountered. When the toolchains are in the Pio registry everything looks fine so far. Managed to compile and flash a simple Blink example.

I decided for my (Arduino Lib builder) build process to include the relevant sdk packages in the framework. So it is 100% save that always the correct SDK lib files are used. Would suggest you provide a package with all needed inside for Platformio. The needed changes in Aruino Lib builder to do this are just a few. Did this quick here https://github.com/Jason2866/esp32-arduino-lib-builder/blob/v5.1/tools/archive-build.sh (Removed some stuff we do not need/use in Project Tasmota...)

me-no-dev commented 1 year ago

@Jason2866 can you post the contents of package.json?

@ivankravets I am getting 504 when trying to upload the new toolchain packages.

pio package publish --non-interactive --owner espressif /Users/ficeto/Desktop/ESP32/pio_packages/out-12.2.0+20230208/toolchain-riscv32-esp-darwin_arm64-12.2.0+20230208.tar.gz
Error: <html>
<head><title>504 Gateway Time-out</title></head>
<body>
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx</center>
</body>
</html>

Preparing a package...
Type:     tool
Owner:    espressif
Name:     toolchain-riscv32-esp
Version:  12.2.0+20230208
System:   darwin_arm64
Size:     86.90MB
The package publishing may take some time depending on your Internet connection and the package size.
Publishing...
pio package publish /Users/ficeto/Desktop/ESP32/pio_packages/out-12.2.0+20230208/toolchain-riscv32-esp-darwin_arm64-12.2.0+20230208.tar.gz failed
ivankravets commented 1 year ago

@me-no-dev , could you try again? We will try to monitor logs. Thanks!

me-no-dev commented 1 year ago

@ivankravets just did. Same result :)

Jason2866 commented 1 year ago

@me-no-dev This is the package.json from the patch5 toolchain for Linux

{
  "name": "toolchain-xtensa-esp32",
  "version": "8.4.0+2021r2-patch5",
  "description": "GCC Toolchain for Espressif ESP32 Xtensa MCUs",
  "keywords": [
    "build tools",
    "compiler",
    "assembler",
    "linker",
    "preprocessor",
    "espressif",
    "esp32"
  ],
  "license": "GPL-2.0-or-later",
  "repository": {
    "type": "git",
    "url": "https://github.com/espressif/crosstool-NG"
  },
  "system": "linux_x86_64"
}

For the 12.2.0 a valid entry is "version": "12.2.0+20230208",

Jason2866 commented 1 year ago

@me-no-dev A lot of error prone work could be saved, (no repacking) if a package.json would be built and added when the toolchains are compiled and released. Suggested this https://github.com/espressif/crosstool-NG/issues/9

me-no-dev commented 1 year ago

ah seems uploads are working now 🎉

Jason2866 commented 1 year ago

@me-no-dev The esp32 Apple Arm toolchain is already online and working :-)

Blink sketch:

 *  Task wird ausgeführt: platformio run 

Processing esp-wrover-kit (framework: arduino; platform: https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5; board: esp-wrover-kit)
--------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp-wrover-kit.html
PLATFORM: Espressif 32 (2023.4.0+sha.32f22d8) > Espressif ESP-WROVER-KIT
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (ftdi) On-board (ftdi) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-arduinoespressif32 @ 3.0.0 
 - tool-esptoolpy @ 1.40501.0 (4.5.1) 
 - tool-mklittlefs @ 1.203.210628 (2.3) 
 - tool-mkspiffs @ 2.230.0 (2.30) 
 - toolchain-xtensa-esp32 @ 12.2.0+20230208
Converting Blink.ino
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 31 compatible libraries
Scanning dependencies...
No dependencies
Building in release mode
Compiling .pio/build/esp-wrover-kit/src/Blink.ino.cpp.o
Building .pio/build/esp-wrover-kit/bootloader.bin
Generating partitions .pio/build/esp-wrover-kit/partitions.bin
esptool.py v4.6-dev
Creating esp32 image...
Merged 1 ELF section
Successfully created esp32 image.
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/Esp.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/FirmwareMSC.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/FunctionalInterrupt.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/HWCDC.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/HardwareSerial.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/IPAddress.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/IPv6Address.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/MD5Builder.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/Print.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/Stream.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/StreamString.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/Tone.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/USB.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/USBCDC.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/USBMSC.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/WMath.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/WString.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/base64.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/cbuf.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-adc.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-bt.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-cpu.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-dac.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-gpio.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-i2c-slave.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-i2c.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-ledc.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-matrix.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-misc.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-periman.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-psram.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-rgb-led.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-rmt.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-sigmadelta.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-spi.c.o
In file included from /Users/hans/.platformio/packages/framework-arduinoespressif32/cores/esp32/esp32-hal-rmt.c:16:
/Users/hans/.platformio/packages/framework-arduinoespressif32/tools/esp32-arduino-libs/esp32/include/driver/deprecated/driver/rmt.h:18:2: warning: #warning "The legacy RMT driver is deprecated, please use driver/rmt_tx.h and/or driver/rmt_rx.h" [-Wcpp]
   18 | #warning "The legacy RMT driver is deprecated, please use driver/rmt_tx.h and/or driver/rmt_rx.h"
      |  ^~~~~~~
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-time.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-timer.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-tinyusb.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-touch.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/esp32-hal-uart.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/firmware_msc_fat.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/libb64/cdecode.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/libb64/cencode.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/main.cpp.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/stdlib_noniso.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/wiring_pulse.c.o
Compiling .pio/build/esp-wrover-kit/FrameworkArduino/wiring_shift.c.o
Archiving .pio/build/esp-wrover-kit/libFrameworkArduino.a
Indexing .pio/build/esp-wrover-kit/libFrameworkArduino.a
Linking .pio/build/esp-wrover-kit/firmware.elf
Retrieving maximum program size .pio/build/esp-wrover-kit/firmware.elf
Checking size .pio/build/esp-wrover-kit/firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [          ]   4.2% (used 13640 bytes from 327680 bytes)
Flash: [=         ]   9.7% (used 126509 bytes from 1310720 bytes)
Building .pio/build/esp-wrover-kit/firmware.bin
esptool.py v4.6-dev
Creating esp32 image...
Merged 2 ELF sections
Successfully created esp32 image.
===================================== [SUCCESS] Took 5.11 seconds =====================================
 *  Das Terminal wird von Aufgaben wiederverwendet, drücken Sie zum Schließen eine beliebige Taste.
valeros commented 1 year ago

Hi @me-no-dev, the espressif/toolchain-riscv32-esp @ 8.4.0+2021r2-patch5 packages have been restored, plesase restart CI if still needed.

me-no-dev commented 1 year ago

@valeros thanks a bunch! CI passes now

Jason2866 commented 1 year ago

Thx!

valeros commented 1 year ago

Hi @me-no-dev , sorry to hijack the already closed issue, I just wanted to answer your question above.

maybe good idea to poke here about going through some of the changes in Arduino v3.0.0 so that we can enable PIO CI again? And that you are aware of what is coming? The main difference is that we now have the SDK as external package installed by Arduino IDE

I skimmed through the esp-idf-v5.1-libs branch (I hope it's the correct one where you're developing v3.0) and it seems that your first option where main platformio-build.py with generic functionality is still distributed with the Arduino Core while target-specific PlatformIO build scripts will be part of the new esp32-arduino-libs is the right choice at the moment.

The only thing that that looks worrying is the version of that new package (e.g. idf-release_v5.1-420ebd208a). Will such versions be used only while core v3.0 is under development? Will it be possible to use something more conventional?

It'd be great to have a version string that follows the SemVer rules, for example idf-release_v5.1-420ebd208a will become 5.1.0+sha.420ebd208a or something similar. This way you will significantly simplify version managing for third-party integrations including PlatformIO.

Thanks in advance. 

me-no-dev commented 1 year ago

@valeros pio package version in only set in the package json, correct? If so, then this should not be an issue and we can name it as you propose for PIO and have it any way that makes sense to us in Arduino. Just please give me an example json that should be inside the package. Also we are thinking of the possibility to split the packages by target.

me-no-dev commented 1 year ago

Overall I think it could be beneficial for both parties if we schedule a short call to come up with a plan. Currently for development we pull a snapshot of the particular branch in esp32-arduino-libs and will repackage it when releasing. PIO would need the release package for releases, but what would you do for development? Can you use the same link without version?

leon0399 commented 2 months ago

@me-no-dev can you, please publish 13.2.0?

Jason2866 commented 2 months ago

@leon0399 The are already there. https://registry.platformio.org/tools/platformio/toolchain-xtensa-esp-elf

leon0399 commented 2 months ago

So this one is now incorrect? https://registry.platformio.org/tools/espressif/toolchain-xtensa-esp32

Jason2866 commented 2 months ago

Name changed of the toolchains