oscarsan1 commented 4 years ago

When building for first time with platformio I get an error every missing library, but every building task Platformio intalls only one more library. The error happens only with web referenced libraries like: https://github.com/marvinroger/async-mqtt-client#v0.8.1 https://github.com/xoseperez/debounceevent.git#2.0.5 https://github.com/xoseperez/eeprom_rotate#0.9.2

but not with other "direct" libraries, that are instaled without error: Brzo I2C ... Embedis

Then I have to run build task 21 times until the build success. Each time one more library is instaled.

The error is in this case with Installing NtpClient, next I build I get the next library error RFM69:

Processing wemos-d1mini-relayshield (board: esp12e; platform: espressif8266@1.5.0; framework: arduino)

Verbose mode can be enabled via -v, --verbose option CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/esp12e.html PLATFORM: Espressif 8266 1.5.0 > Espressif ESP8266 ESP-12E HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash PACKAGES: toolchain-xtensa 1.40802.0 (4.8.2), framework-arduinoespressif8266 1.20300.1 (2.3.0), tool-esptool 1.409.0 (4.9) Converting espurna.ino LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf LDF Modes: Finder ~ chain, Compatibility ~ soft LibraryManager: Installing NtpClient git version 2.24.0.windows.2 Cloning into 'c:_proyecto_Espurna\espurna-master 1.14.0\code.pio\libdeps\wemos-d1mini-relayshield_tmp_installing-wmpzjmhu-package'... HEAD is now at 0942ebc Merge remote-tracking branch 'upstream/develop' into develop PermissionError: [WinError 5] Acceso denegado: 'c:\_proyecto\_Espurna\espurna-master 1.14.0\code\.pio\libdeps\wemos-d1mini-relayshield\_tmp_installing-wmpzjmhu-package\.git\objects\pack\pack-efbfd8063d05b562ed75cf3b48da369c59a5fe32.idx': File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\builder\main.py", line 151: env.SConscript("$BUILD_SCRIPT") File "C:\Users\Osr.platformio\packages\tool-scons\script..\engine\SCons\Script\SConscript.py", line 605: return _SConscript(self.fs, *files, subst_kw) File "C:\Users\Osr.platformio\packages\tool-scons\script..\engine\SCons\Script\SConscript.py", line 286: exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals) File "C:\Users\Osr.platformio\platforms\espressif8266@1.5.0\builder\main.py", line 375: target_elf = env.BuildProgram() File "C:\Users\Osr.platformio\packages\tool-scons\script..\engine\SCons\Environment.py", line 224: return self.method(*nargs, *kwargs) File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\builder\tools\platformio.py", line 140: _build_project_deps(env) File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\builder\tools\platformio.py", line 48: project_lib_builder = env.ConfigureProjectLibBuilder() File "C:\Users\Osr.platformio\packages\tool-scons\script..\engine\SCons\Environment.py", line 224: return self.method(nargs, kwargs) File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\builder\tools\piolib.py", line 1027: project.install_dependencies() File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\builder\tools\piolib.py", line 859: lm.install(uri) File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\managers\lib.py", line 301: force=force, File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\managers\package.py", line 696: pkg_dir = self._install_from_url(name, url, requirements, track=True) File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\managers\package.py", line 497: return self._install_from_tmp_dir(_tmp_dir, requirements) File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\managers\package.py", line 583: shutil.move(tmp_dir, pkg_dir) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 578: rmtree(src) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 516: return _rmtree_unsafe(path, onerror) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 395: _rmtree_unsafe(fullname, onerror) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 395: _rmtree_unsafe(fullname, onerror) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 395: _rmtree_unsafe(fullname, onerror) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 400: onerror(os.unlink, fullname, sys.exc_info()) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 398: os.unlink(fullname) ===================================================== [FAILED] Took 5.12 seconds =====================================================

Next build i get some direct libraries instaled and next library error:

Processing wemos-d1mini-relayshield (board: esp12e; platform: espressif8266@1.5.0; framework: arduino)

Verbose mode can be enabled via -v, --verbose option CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/esp12e.html PLATFORM: Espressif 8266 1.5.0 > Espressif ESP8266 ESP-12E HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash PACKAGES: toolchain-xtensa 1.40802.0 (4.8.2), framework-arduinoespressif8266 1.20300.1 (2.3.0), tool-esptool 1.409.0 (4.9) Converting espurna.ino LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf LDF Modes: Finder ~ chain, Compatibility ~ soft Looking for OneWire library in registry Conflict: More than one library has been found by request {"name": "OneWire", "requirements": null}: OneWire Automatically chose the first available library (use --interactive option to make a choice)=======

ID: 1

Control 1-Wire protocol (DS18S20, DS18B20, DS2408 and etc)

Keywords: onewire, temperature, bus, 1-wire, ibutton, sensor Compatible frameworks: Arduino Compatible platforms: Authors: Paul Stoffregen, Jim Studt, Tom Pollard, Derek Yerger, Josh Larios, Robin James, Glenn Trewitt, Jason Dangel, Guillermo Lovato, Ken Butcher, Mark Tillotson, Bertrik Sikken, Scott Roberts

OneWire

ID: 2604

OneWire Library lets you access 1-wire devices made by Maxim/Dallas, such as temperature sensors

Keywords: 1-wire, ds18b20, one, one wire, onewire Compatible frameworks: mbed Compatible platforms: Atmel SAM, Freescale Kinetis, Maxim Integrated MAX32, Nordic nRF51, Nordic nRF52, NXP LPC, Silicon Labs EFM32, ST STM32, Teensy, WIZNet W7500 Authors: Ivan Kravets

OneWire

ID: 2603

Dallas' 1-Wire bus protocol library

Keywords: 1-wire, one wire, one-wire Compatible frameworks: mbed Compatible platforms: Atmel SAM, Freescale Kinetis, Maxim Integrated MAX32, Nordic nRF51, Nordic nRF52, NXP LPC, Silicon Labs EFM32, ST STM32, Teensy, WIZNet W7500 Authors: Zoltan Hudak

OneWire

ID: 4337

onewire 1-wire ds18x20 ds2450 multi-channel

Keywords: 1-wire, ds18x20, ds2450, multi-channel, onewire Compatible frameworks: mbed Compatible platforms: Atmel SAM, Freescale Kinetis, Maxim Integrated MAX32, Nordic nRF51, Nordic nRF52, NXP LPC, Silicon Labs EFM32, ST STM32, Teensy, WIZNet W7500 Authors: frederic blanc

OneWire

ID: 2602

Dallas 1-wire driver with DS18B20 temperature sensor support; custom bus driver to work with https://emir.googlecode.com/svn/emir2/trunk/eagle/emir-shield.sch

Keywords: 1-wire, 1wire, ds18b20, ds18s20, onewire Compatible frameworks: mbed Compatible platforms: Atmel SAM, Freescale Kinetis, Maxim Integrated MAX32, Nordic nRF51, Nordic nRF52, NXP LPC, Silicon Labs EFM32, ST STM32, Teensy, WIZNet W7500 Authors: Ales Povalac

OneWire

ID: 4340

Implemets a 1-Wire library, currently without "Search ROM" support

Keywords: 1, 1-wire, ds1820, maxim, one, onewire, wire Compatible frameworks: mbed Compatible platforms: Atmel SAM, Freescale Kinetis, Maxim Integrated MAX32, Nordic nRF51, Nordic nRF52, NXP LPC, Silicon Labs EFM32, ST STM32, Teensy, WIZNet W7500 Authors: Jan Achterhold

OneWire

ID: 4341

An experimental timer interrupt driven software based 1-Wire master interface that was created by Robert David Brinzer for a University of Glasgow Level 4 Electronic and Software Engineering Final Year ...

Keywords: 1-wire, dallas, maxim, onewire Compatible frameworks: mbed Compatible platforms: Atmel SAM, Freescale Kinetis, Maxim Integrated MAX32, Nordic nRF51, Nordic nRF52, NXP LPC, Silicon Labs EFM32, ST STM32, Teensy, WIZNet W7500 Authors: Scott Roy

OneWire

ID: 4339

DS18B20

Keywords: onewire Compatible frameworks: mbed Compatible platforms: Atmel SAM, Freescale Kinetis, Maxim Integrated MAX32, Nordic nRF51, Nordic nRF52, NXP LPC, Silicon Labs EFM32, ST STM32, Teensy, WIZNet W7500 Authors: jack zen

Found: https://platformio.org/lib/show/1/OneWire LibraryManager: Installing id=1 OneWire @ 2.3.5 has been successfully installed! Looking for PZEM004T library in registry Found: https://platformio.org/lib/show/1662/PZEM004T LibraryManager: Installing id=1662 PZEM004T @ 1.1.2 has been successfully installed! Installing dependencies Looking for EspSoftwareSerial library in registry Found: https://platformio.org/lib/show/168/EspSoftwareSerial LibraryManager: Installing id=168 @ >=3.2.0 EspSoftwareSerial @ 6.4.0 has been successfully installed! Looking for PubSubClient library in registry Found: https://platformio.org/lib/show/89/PubSubClient LibraryManager: Installing id=89 PubSubClient @ 2.7 has been successfully installed! Looking for rc-switch library in registry Found: https://platformio.org/lib/show/246/rc-switch LibraryManager: Installing id=246 rc-switch @ 2.6.2 has been successfully installed! LibraryManager: Installing RFM69 git version 2.24.0.windows.2 Cloning into 'c:_proyecto_Espurna\espurna-master 1.14.0\code.pio\libdeps\wemos-d1mini-relayshield_tmp_installing-6rc7b1lv-package'... Note: switching to 'cb68f71ae81adb5801d197d7835065d7e1f1e125'.

You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example:

git switch -c

Or undo this operation with:

git switch -

Turn off this advice by setting config variable advice.detachedHead to false

PermissionError: [WinError 5] Acceso denegado: 'c:\_proyecto\_Espurna\espurna-master 1.14.0\code\.pio\libdeps\wemos-d1mini-relayshield\_tmp_installing-6rc7b1lv-package\.git\objects\00\0e2201ae41ca43edd772806078dc2d71b03b52': File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\builder\main.py", line 151: env.SConscript("$BUILD_SCRIPT") File "C:\Users\Osr.platformio\packages\tool-scons\script..\engine\SCons\Script\SConscript.py", line 605: return _SConscript(self.fs, *files, subst_kw) File "C:\Users\Osr.platformio\packages\tool-scons\script..\engine\SCons\Script\SConscript.py", line 286: exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals) File "C:\Users\Osr.platformio\platforms\espressif8266@1.5.0\builder\main.py", line 375: target_elf = env.BuildProgram() File "C:\Users\Osr.platformio\packages\tool-scons\script..\engine\SCons\Environment.py", line 224: return self.method(*nargs, *kwargs) File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\builder\tools\platformio.py", line 140: _build_project_deps(env) File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\builder\tools\platformio.py", line 48: project_lib_builder = env.ConfigureProjectLibBuilder() File "C:\Users\Osr.platformio\packages\tool-scons\script..\engine\SCons\Environment.py", line 224: return self.method(nargs, kwargs) File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\builder\tools\piolib.py", line 1027: project.install_dependencies() File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\builder\tools\piolib.py", line 859: lm.install(uri) File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\managers\lib.py", line 301: force=force, File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\managers\package.py", line 696: pkg_dir = self._install_from_url(name, url, requirements, track=True) File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\managers\package.py", line 497: return self._install_from_tmp_dir(_tmp_dir, requirements) File "C:\Users\Osr.platformio\penv\lib\site-packages\platformio\managers\package.py", line 583: shutil.move(tmp_dir, pkg_dir) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 578: rmtree(src) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 516: return _rmtree_unsafe(path, onerror) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 395: _rmtree_unsafe(fullname, onerror) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 395: _rmtree_unsafe(fullname, onerror) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 395: _rmtree_unsafe(fullname, onerror) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 400: onerror(os.unlink, fullname, sys.exc_info()) File "C:\Users\Osr.platformio\python37\lib\shutil.py", line 398: os.unlink(fullname)

oscarsan1 commented 4 years ago

I also found error with missing libraries:

SPIFlash
DNSServer

After I include this libraries to the lib_deps in the platformio.ini finaly build without errors.

mcspr commented 4 years ago

Does it change anything if you use global library storage?

diff --git a/code/scripts/pio_pre.py b/code/scripts/pio_pre.py
index adcdc13b..c8b63546 100644
--- a/code/scripts/pio_pre.py
+++ b/code/scripts/pio_pre.py
@@ -93,4 +93,4 @@ if os.environ.get("ESPURNA_PIO_SHARED_LIBRARIES"):
         storage = get_shared_libdeps_dir("common", "shared_libdeps_dir")
         print("using shared library storage: ", storage, file=sys.stderr)

-    subprocess_libdeps(env.GetProjectOption("lib_deps"), storage)
+subprocess_libdeps(env.GetProjectOption("lib_deps"), None)

This will install things to ~/.platformio/lib

SPIFlash
DNSServer

Are builtin libraries, there is no need to add them unless you have modified .ini even more while testing this and now use a different library dependency finder.

I am not sure what exactly changed in PIO 4.1.0, but this is not the first report of broken library installs with git: https://github.com/xoseperez/espurna/issues/2006

I'd post this to the https://github.com/platformio/platformio-core instead, with some example project like this:

platformio.ini

[env:d1_mini]
platform = espressif8266
framework = arduino
board = d1_mini
lib_deps =
https://github.com/marvinroger/async-mqtt-client#v0.8.1
https://github.com/me-no-dev/ESPAsyncTCP#7e9ed22
https://github.com/me-no-dev/ESPAsyncWebServer#b0c6144

src/main.cpp
```
#include <Arduino.h>
#include <ESPAsyncTCP.h>
#include <AsyncMqttClient.h>
#include <ESPAsyncWebServer.h>
void setup() { }
void loop() { }
```
With a couple of lib_deps pointing to git. Funny thing is installation is actually successful, but the temporary file removal is broken because of either VSCode or some antivirus solution.

xoseperez / espurna

Platformio lib_deps instaling error #2062

Processing wemos-d1mini-relayshield (board: esp12e; platform: espressif8266@1.5.0; framework: arduino)

Processing wemos-d1mini-relayshield (board: esp12e; platform: espressif8266@1.5.0; framework: arduino)

ID: 1

OneWire

ID: 2604

OneWire

ID: 2603

OneWire

ID: 4337

OneWire

ID: 2602

OneWire

ID: 4340

OneWire

ID: 4341

OneWire

ID: 4339