Custom Git platform_packages not fetched correctly

rohansingh commented 3 years ago

What kind of issue is this?

[X] PlatformIO Core. If you’ve found a bug, please provide an information below.

Configuration

Operating system: macOS

PlatformIO Version 5.0.2a2

Description of problem

Specifying a custom Git URL in platform_packages sometimes doesn't work. The specified Git repo appears to be cloned, but ~/.platformio/packages/framework-X does not end up with the correct files. Instead, it ends up containing a few files for something called json-patch-test-suite.

Steps to Reproduce

Use an env like this:

[env]
platform = espressif32
framework = arduino, espidf
platform_packages =
  framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#idf-release/v4.0
  framework-espidf @ https://github.com/espressif/esp-idf.git

Run pio run.

Actual Results

Based on CLI output, both frameworks above appear to be cloned. The Arduino framework appears to end up in the correct spot, at ~/.platformio/packages/framework-arduinoespressif32.

However, the ESP-IDF framework disappears into the ether. The ~/.platformio/packages/framework-espidf directory ends up with a scant few files:

[~/.platformio/packages/framework-espidf]$ ls
README.md              cjson-utils-tests.json package.json           spec_tests.json        tests.json             version.txt

The contents of the package.json and README in there indicate that this is something called "json-patch-test-suite". No idea how it got there.

Expected Results

Expect a copy of the ESP-IDF from the specified Git URL to end up in the right place.

Additional info

Using git@github.com:espressif/esp-idf.git instead of a HTTPS URL still has the same issue. Using my own ESP-IDF fork (tidbyt/esp-idf) has the same issue.

Notably, this all worked fine before upgrading to PlatformIO Core v5. Not sure if #3624 is related.

If problems with PlatformIO Build System:

The content of platformio.ini:

[env]
platform = espressif32
framework = arduino, espidf
platform_packages =
  framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#idf-release/v4.0
  framework-espidf @ https://github.com/espressif/esp-idf.git
board_build.cmake_extra_args = -DPYTHON=python3
board_build.embed_files =
  src/assets/ble.webp
  src/assets/boot.webp
  src/assets/connect.webp
  src/assets/noapps.webp
  src/assets/setup.webp
extra_scripts =
  pre:extra_scripts/pre.py
test_build_project_src = yes
upload_port = /dev/cu.SLAB_USBtoUART

[env:tidbyt-v7]
board = tidbyt-v7
monitor_speed = 115200
monitor_dtr = 0
monitor_rts = 0
monitor_flags = --raw
build_flags =
  -DTIDBYT_PIN_LAYOUT_VERSION=6
  -DTIDBYT_SCL_GPIO=0
  -DTIDBYT_SDA_GPIO=3

ivankravets commented 3 years ago

framework-espidf @ https://github.com/espressif/esp-idf.git

This is invalid package for PlatformIO. We support only stable ESP-IDF.

rohansingh commented 3 years ago

That was just an example. It fails in the same way with my fork that's based on the ESP-IDF stable and just has a couple patches.

Either way, isn't it concerning that some weird files are ending up in packages/framework-espidf rather than the actual Git ref specified?

ivankravets commented 3 years ago

Does that package contain package.json manifest? Otherwise, PlatformIO will search for it recursively within archive/repo.

rohansingh commented 3 years ago

Ah, It does not contain a package.json, that must be the issue. Not sure how it worked with PlatformIO v4.3.4. I'll fix it, thanks so much!

Just curious, are the scripts that generate the official framework-espidf available somewhere? It would be really useful to be able to see exactly how an upstream ESP-IDF rev becomes a PlatformIO framework-espidf release.

ivankravets commented 3 years ago

We just use source code from https://github.com/espressif/esp-idf/tags.

You can fork ESP-IDF and add this package.json.

platformio / platformio-core