Build process fails while PIO is rebuilding IntelliSense index

[blazoncek]

I am seeing plenty of failed builds recently and the only cause I can put a finger on is rebuilding IntelliSense index.

When I hit Build (menu or button), PIO starts build process but after a second or two a "PlatformIO: Rebuild IntelliSense index" appears in the status line of VSC. At that point build process fails. If I keep pressing build (after a failed one) I will eventually succeed in building my project. At this point I may get a message "Error: Multiple requests to rebuild ..." and after this message I have a window of 10 min when all builds succeed.

There were no changes in my platformio.ini or platformio_override.ini files and no included libraries were updated.

Sample error messages:

[blazoncek]

I also noticed that it tries to compile every source file even though no changes were made to them.

[ivankravets]

Where did you get this project? Please share a link.

[blazoncek]

Currently working on 2D support for WLED. https://github.com/blazoncek/WLED/tree/2D-support

[blazoncek]

It is also happening to other users (there were complaints on WLED Discord) using upstream repo. This has only been happening since Tuesday this week.

[ivankravets]

Please forward your question to the https://community.platformio.org/ Our awesome community is glad to help you.

[blazoncek]

Ok. 😄 Will do, thought I see no connection with the community. There were no changes from Monday to Tuesday neither to code or libraries, I did not update PIO or VSC and out of blue 7 out of 10 compiles fail with varying error messages (mostly file not found).

Let me repeat: Every time I hit Build PIO will start rebuilding IntelliSense index. Even if I hit Build right one after another. And every build compiles all source files (though actual compile fetches cached object files).

I will try do delete .pio folder (and possibly ~/.platformio) and see how it goes.

[tonyn0]

Just piling on with the issue... ;)

[ivankravets]

Please check your lib_deps in configuration file. It seems PIO reinstalls dependencies on each build.

[blazoncek]

New info: when hitting Build whole content of .pio/build folder gets erased. All environments even those not touched by build. I always build a single environment from PIO environment tree. Lib_deps are untouched and most of them at exact version.

I also noticed that some of ESP8266 environments are lacking Platform menu.

[blazoncek]

This would be my custom environment from platformio_oveeride.ini:

[env:d1_mini_temp]
extends = env:d1_mini ;use defaults from master
board_build.filesystem = littlefs
upload_port = /dev/cu.wchusbserial1a136120
build_flags = ${common.build_flags_esp8266}
  -D WLED_DISABLE_BLYNK
  -D WLED_DISABLE_CRONIXIE
  -D WLED_DISABLE_HUESYNC
  -D USERMOD_FOUR_LINE_DISPLAY
  -D USERMOD_DALLASTEMPERATURE
  -D TEMPERATURE_PIN=13 # (D7)
  -D LEDPIN=2     # (D4)
  -D RLYPIN=12    # (D6)
  -D BTNPIN=0     # (D3)
  -D IRPIN=14     # (D5)
  -D USERMOD_MULTI_RELAY
  -D USERMOD_PIRSWITCH
  -UWLED_USE_MY_CONFIG
lib_deps = ${esp8266.lib_deps}
  U8g2@~2.28.8
  OneWire@~2.3.5
  Wire

[env:d1_mini_custom_debug]
extends = env:d1_mini_temp
board_build.filesystem = littlefs
monitor_port = /dev/cu.wchusbserial1a136120
monitor_speed = 115200
build_flags = ${common.build_flags_esp8266}
  -D USERMOD_FOUR_LINE_DISPLAY
;  -D USERMOD_AUTO_SAVE
  -D USERMOD_PIRSWITCH
  -D PIR_SENSOR_PIN=16 # (D0) on D1mini
  -D USERMOD_DALLASTEMPERATURE
  -D TEMPERATURE_PIN=13 # (D7) [4 (D2) Wemos DS18B20 shield]
  -D LEDPIN=2     # (D4)
# -D DATAPIN=13   # (D7) MOSI
# -D CLOCKPIN=14  # (D5) SCLK
# -D LED_TYPE=50  # TYPE_WS2801 (from const.h)
  -D BTNPIN=0     # (D3)
  -D RLYPIN=12    # (D6) [5 (D1) Wemos Relay Shield]
  -D IRPIN=14     # (D5)
;  -D USERMOD_MULTI_RELAY
;  -D MULTI_RELAY_MAX_RELAYS=1
  -D USERMOD_ANIMATED_STAIRCASE
  -D WLED_DEBUG
  -UWLED_USE_MY_CONFIG

esp8266.lib_deps

lib_deps = 
  ${env.lib_deps}
  #https://github.com/lorol/LITTLEFS.git
  ESPAsyncTCP @ 1.2.2
  ESPAsyncUDP
  makuna/NeoPixelBus @ 2.6.9

env.lib_deps

lib_deps =
    fastled/FastLED @ 3.5.0
    IRremoteESP8266 @ 2.8.2
    https://github.com/Aircoookie/ESPAsyncWebServer.git @ ~2.0.4
  #For use of the TTGO T-Display ESP32 Module with integrated TFT display uncomment the following line
    #TFT_eSPI
  #For use SSD1306 OLED display uncomment following
    #U8g2@~2.28.8
    #U8g2@~2.32.10
  #For Dallas sensor uncomment following 2 lines
    #OneWire@~2.3.5
    #milesburton/DallasTemperature@^3.9.0
  #For BME280 sensor uncomment following
    #BME280@~3.0.0
    ; adafruit/Adafruit BMP280 Library @ 2.1.0
    ; adafruit/Adafruit CCS811 Library @ 1.0.4
    ; adafruit/Adafruit Si7021 Library @ 1.4.0

[blazoncek]

Please check your lib_deps in configuration file. It seems PIO reinstalls dependencies on each build.

Dependencies are not rebuilt each time. Double checked. What does change each time I hit Build is whole .pio/build folder content is erased. This also explains why it always tries to compile every source file.

[ivankravets]

Do you change file structure in the src folder? Do you run any extra scripts?

[blazoncek]

No. Nothing has changed (except source files themselves) for quite some time now. Must be months. The newest timestamp in src folder is 10.5.2022 but that is only on my notebook my desktop has files dated 7.12.2021.

[tonyn0]

It's taking seven Build cycles to finally compile.

[henrygab]

Also seeing this on an unrelated project ESPixelStick.

I get ~95% failure rates when building a single environment repeatedly. The error is nearly always the *** SRC\.pio\build\ENV\.sconsign39.dblite: No such file or directory error.

I confirm that the entire .pio/build folder content is erased for each build.

I confirm that the IntelliSense index rebuild gets triggered every build.

When I build multiple environments at once (e.g., "default"), the first environment built has the same failure rate as above. The remaining environments have ~70% success rate ... sometimes failing with the .sconsign39.dblite error, sometimes failing when linking, sometimes saying source files are missing during compilation.

NOTE: These errors occur, even if I temporarily disable antivirus real-time protection (Windows Defender Antivirus).

[ivankravets]

A lot of similar issues on the community forum. People think they are protected by “extra antivirus tools”, but that is not true. These tools make more problem for them.

[blazoncek]

I am not using any antivirus or other malware tools. Using macOS Mojave & High Sierra.

[ivankravets]

Could you provide a simple project to reproduce this issue? You can keep the configuration file without changes and remove the src code. Pack as zip archive and attach here. Please verify that this simple example reproduce this issue.

Thanks.

[henrygab]

Two entirely unrelated projects:

https://github.com/forkineye/ESPixelStick/archive/refs/heads/main.zip https://github.com/Aircoookie/WLED/archive/refs/heads/main.zip

The ESPixelStick is more simple than the WLED project.

Steps to repro with the ESPixelStick project:

1. clone the repo, or download from above link.
2. build the "espsv3" environment once (first time might work)
3. modify ./ESPixelStick/src/ESPixelStick.h (change only the comment... no code change required)
4. build the "default" environment

Other additional repro scenarios:

5. wait 15+ minutes from last work (allow indexer to restart) 6. Modify one source file and rebuild the "default" environment again 7. wait 15+ minutes from last work (allow indexer to restart) 8. Modify platformio.ini to have following `default_envs`: ```ini [platformio] default_envs = espsv3, esp32_quinled_quad_eth ``` 9. Modify one source file and rebuild "default" at least six times within three minutes

At step 4, notice at least one .sconsign39.dblite error, and that PlatformIO IDE extension is flagged for multiple requests to rebuild the project Intellisense index (preventing it for the next 10 minutes).

Once you have a repro, it seems more efficient for you to remove / reduce aspects as needed, as your knowledge of PIO internals would guide you to better theories as to what is a potential cause, and thus which code paths to try to exclude to see if it continues to happen. In contrast, my attempts to "reduce" the repro would be as a person who is blind, trying to determine which colors

Issue appears to be cross-platform

You yourself wrote: > A lot of similar issues on the community forum. People think they are protected by “extra antivirus tools”, but that is not true. These tools make more problem for them. * I have experienced the errors on Windows, Blaž on two MacOS versions. * I have disabled real-time AV (the built-in AV from Microsoft, not 3rd party AV with more problematic filters), and the issue still occurred. * You also noted many similar issues on the community forum. * The `.sconsign39.dblite` error is **_not a new error_**; it just happens much more frequently now, so the pain is higher. In combination, the above seems to suggest that a single root cause is likely, and that neither the OS nor anti-virus are themselves the cause, because they are not the common aspects of the bug report.

[blazoncek]

Henry did provide excellent instructions. I can confirm that when you overload IntelliSense Index rebuild process (with multiple build requests in quick succession, so that above error message appears) build process will be always successful until IntelliSense Index rebuild process is restarted after 10 minutes.

The process fails while (or immediately after) "Scanning dependencies..." most of the time and it can be seen from the status line that IntelliSense Index rebuilding starts then.

I am having more success with a custom platformio_override.ini and first of the two default environments:

[platformio]
default_envs = wemos_d1_mini32_debug, d1_mini_custom_debug

[noisymime]

Another +1 on this issue, specifically related to building for PIO unit tests.

OS: MacOS Monterey 12.4 VSCode: 1.67.2 PlatformIO: 6.0.1

• If I run the PIO unit tests through VSCode I get many almost randomNo such file or directory errors as well as a few scons ones. Run the tests again and the 'missing' files will be different
• If I run pio test from a separate terminal when VSCode is running in the background, I get the same errors
• If I run that same pio test from a terminal when VSCode is NOT running, everything works fine.

This is a relatively new thing, these tests were running fine in VSCode previously, though I can't pinpoint exactly when the problem began.

[ivankravets]

@noisymime do you use any extra scripts in the project?

[noisymime]

@noisymime do you use any extra scripts in the project?

No, none at all.

[ivankravets]

Could someone provide a remote session under your control to debug this issue? Please mail to me@ikravets.com and I'll provide further details. Thanks!

[JohnScottUK]

This appears to be related to an update to VS Code. With 1.67 (April 2022) I see repeated PlatformIO rebuilding index and eventually the following error. With VSCode 1.66 the problem was never observed.

Error: Multiple requests to rebuild the project "t800-hk" index have been received! Automatic index rebuilding process has been terminated for 10 minutes. at A.value (/Users/johnscott/.vscode/extensions/platformio.platformio-ide-2.4.3/node_modules/platformio-node-helpers/dist/index.js:1:790605) at A.value (/Users/johnscott/.vscode/extensions/platformio.platformio-ide-2.4.3/node_modules/platformio-node-helpers/dist/index.js:1:792714) at A.value (/Users/johnscott/.vscode/extensions/platformio.platformio-ide-2.4.3/node_modules/platformio-node-helpers/dist/index.js:1:794550) at /Users/johnscott/.vscode/extensions/platformio.platformio-ide-2.4.3/node_modules/platformio-node-helpers/dist/index...

[ivankravets]

@JohnScottUK could you remove the .pio folder from a project and restart VSCode. Does it still claim "Multiple requests"?

[JohnScottUK]

@JohnScottUK could you remove the .pio folder from a project and restart VSCode. Does it still claim "Multiple requests"?

Closed VSCode; Deleted .pio; Restarted VSCode. Same error eventually returns. I see the following in the status bar repeating:

[ivankravets]

Could you provide a quick session to debug this issue? Please email me@ikravets.com.

[JohnScottUK]

Could you provide a quick session to debug this issue? Please email me@ikravets.com.

Emailed.

[henrygab]

I have also emailed, and have setup a fresh Win11 VM that reproduces this issue. Working to provide Ivan the access he requested via the tool he requested.

[ivankravets]

It seems we have found a problem.

1. Please open PlatformIO Core CLI
2. Type pio upgrade --dev
3. Restart VSCode.

Does it work now?

[tonyn0]

Working for me!

[blazoncek]

Looks promising. Build work, compile compiles only changed files,... 👍

[noisymime]

Working now for me also. Thanks for looking into this one!

[henrygab]

:tada: Works for me also! I'm thrilled to see builds failing due to my mistakes! :tada:

Was it this innocuous API causing all the trouble? Who knew that passing the flag exist_ok=True would forcibly remove an existing directory if it existed? That's not clear from the Python documentation.

Now presuming that this is the fix: Well done tracking this one down!

[ivankravets]

Thanks for the confirmation! The PIO Core 6.0.2 is planned for release today/tomorrow.