Newer library is not listed as available

arduino / arduino-cli

Arduino command line tool

https://arduino.github.io/arduino-cli/latest/

GNU General Public License v3.0

4.32k stars 373 forks source link

Newer library is not listed as available #1041

Closed alranel closed 2 years ago

alranel commented 3 years ago

Bug Report

Current behavior

alranel@MacBook-Pro libraries % arduino-cli lib update-index
Updating index: library_index.json downloaded
alranel@MacBook-Pro libraries % arduino-cli lib list
Name                               Installed Available Location Description
Adafruit-PN532-master              1.0.2 -       user     Arduino library for SPI and I2C acces...
Adafruit_BusIO                     1.3.3 -       user     This is a library for abstracting awa...
Adafruit_Circuit_Playground        1.10.4 -       user     All in one library to control Adafrui...
Adafruit_DotStar                   1.1.4 -       user     Adafruit DotStar LED Library
Adafruit_GFX_Library               1.9.0 -       user     Adafruit GFX graphics core library, t...
Adafruit_seesaw_Library            1.3.1 -       user     This is a library for the Adafruit se...
Adafruit_ST7735_and_ST7789_Library 1.5.18 -       user     This is a library for the Adafruit ST...
Arduino_APDS9960                   1.0.3 -       user     A library for the APDS9960 sensor
Arduino_BQ24195                    0.9.0 -       user     Arduino library for the BQ24195 Power...
Arduino_HTS221                     1.0.0 -       user     Allows you to read the temperature an...
Arduino_LPS22HB                    1.0.0 -       user     Allows you to read the pressure senso...
Arduino_LSM6DS3                    1.0.0 -       user     Allows you to read the accelerometer ...
Arduino_MCHPTouch                  1.0.0 -       user     A library for the Atmel touch sensors
Arduino_MicroPython                1.0.0 -       user     MicroPython for Arduino
Arduino_MKRENV                     1.1.0 -       user     Allows you to read the temperature, h...
Arduino_MKRIoTCarrier              0.9.4 -       user     Controlling the IoT MKR Carrier
ArduinoBLE                         1.1.2 1.1.3   user     Enables BLE connectivity on the Ardui...
cie-PN532-master                   1.0.0 -       user     Arduino library for SPI access to the...
MAX77650-Arduino-Library           1.0.1 -       user     An Arduino library for the Maxim Powe...
SD                                 1.2.4 -       user     Enables reading and writing on SD cards.
Servo                              1.1.6 1.1.7   user     Allows Arduino/Genuino boards to cont...
SpacebrewYun                       1.0.2 -       user     Enables the communication between int...
WiFiNINA                           1.7.1 -       user     Enables network connection (local and...

The installed version of Adafruit_ST7735_and_ST7789_Library is 1.5.18 and no newer version is listed under "Available".

Expected behavior

As of now, the library index contains the version 1.6.0 of Adafruit ST7735 and ST7789 Library, so I would expect it to be listed.

This is correctly reflected in the IDE:

Environment

CLI version (output of arduino-cli version): 0.13.0 Commit: 693a045eea420c29ca7027e668eee31bce37365d
OS and platform: macOS 10.15.4

alranel commented 3 years ago

Upgrading the library works though:

alranel@MacBook-Pro libraries % arduino-cli lib install "Adafruit ST7735 and ST7789 Library"
Adafruit ST7735 and ST7789 Library depends on Adafruit BusIO@1.5.0
Adafruit ST7735 and ST7789 Library depends on Adafruit ST7735 and ST7789 Library@1.6.0
Adafruit ST7735 and ST7789 Library depends on Adafruit GFX Library@1.10.2
Adafruit ST7735 and ST7789 Library depends on Adafruit seesaw Library@1.3.1
Adafruit ST7735 and ST7789 Library depends on SD@1.2.4
Downloading Adafruit BusIO@1.5.0...
Adafruit BusIO@1.5.0 downloaded
Installing Adafruit BusIO@1.5.0...
Replacing Adafruit_BusIO@1.3.3 with Adafruit BusIO@1.5.0...
Installed Adafruit BusIO@1.5.0
Downloading Adafruit ST7735 and ST7789 Library@1.6.0...
Adafruit ST7735 and ST7789 Library@1.6.0 downloaded
Installing Adafruit ST7735 and ST7789 Library@1.6.0...
Replacing Adafruit_ST7735_and_ST7789_Library@1.5.18 with Adafruit ST7735 and ST7789 Library@1.6.0...
Installed Adafruit ST7735 and ST7789 Library@1.6.0
Downloading Adafruit GFX Library@1.10.2...
Adafruit GFX Library@1.10.2 downloaded
Installing Adafruit GFX Library@1.10.2...
Replacing Adafruit_GFX_Library@1.9.0 with Adafruit GFX Library@1.10.2...
Installed Adafruit GFX Library@1.10.2
Downloading Adafruit seesaw Library@1.3.1...
Adafruit seesaw Library@1.3.1 already downloaded
Installing Adafruit seesaw Library@1.3.1...
Already installed Adafruit seesaw Library@1.3.1
Downloading SD@1.2.4...
SD@1.2.4 already downloaded
Installing SD@1.2.4...
Already installed SD@1.2.4
alranel@MacBook-Pro libraries % arduino-cli lib list | grep Adafruit_ST7735_and_ST7789_Library
Adafruit_ST7735_and_ST7789_Library 1.6.0 -       user     This is a library for the Adafruit ST...

Please note that I supplied the spaced library name to arduino-cli lib install (Adafruit ST7735 and ST7789 Library) instead of the underscored library name that I see in arduino-cli lib list (Adafruit_ST7735_and_ST7789_Library). By the way, this underscored library name does not reflect any field of the library index so it could be a good idea to document clearly what is a valid "library name" for the arduino-cli commands (maybe worth opening a separate issue).

paulsiersma commented 3 years ago

I can confirm this is a problem on Windows 10 on cli version 0.14.0. I also suspect spaces in the "real name" of the library to play a role in this. But running install instead of upgrade has the result of the latest version being installed.

ubidefeo commented 3 years ago

@paulsiersma when you have spaces in the library name you need to enclose the name in quotes, and it needs to be the exact name (not sure about the casing)

paulsiersma commented 3 years ago

@ubidefeo Thank for your comment, but the arduino-cli lib upgrade command should upgrade all installed libs to latest available versions. That's not working (the newer version is not even listed as being available), and that's what this Issue is about.

ubidefeo commented 3 years ago

@paulsiersma in order to show the latest library version an update is required. When you run operations such as core and library search the index files are updated and new versions of libraries and cores become available through each index. If you simply run arduino-cli lib upgrade "Adafruit ST7735 and ST7789 Library" (assuming you have an older version) then the CLI will download the index for libraries, detect if there's a new one and install it if there is. It seems very strange that it doesn't happen if a new version is available.

@alranel mentions he used the spaced name which is correct, search happens against the name string in the library.properties but the lib list command lists the folder names instead. This might be a misbehaviour requiring an adjustment.

I haven't been able to reproduce this issue, but maybe it's Windows only

josefwegner commented 2 years ago

I can easily reproduce this issue on Raspberry OS (Debian Buster):

Use Arduino IDE's "Manage Libraries..." to downgrade a library with space and one without a space in its real_name. I chose "Adafruit NeoPixel" and "WiFiNINA".
Close Arduino IDE

Run arduino-cli update:

$ arduino-cli update
Updating index: package_index.json downloaded
Updating index: package_index.json.sig downloaded
Updating index: package_damellis_attiny_index.json downloaded
Updating index: package_drazzy.com_index.json downloaded
Updating index: package_MCUdude_MicroCore_index.json downloaded
Updating index: package_MCUdude_MiniCore_index.json downloaded
Updating index: package_MCUdude_MightyCore_index.json downloaded
Updating index: package_nRF5_boards_index.json downloaded
Updating index: package_ehajo_index.json downloaded
Updating index: library_index.json.gz downloaded
Updating index: library_index.json.sig downloaded

Run arduino-cli lib list:

$ arduino-cli lib list                                                                                                                                                                                                                     Name              Installed Available Location              Description
Adafruit_NeoPixel 1.9.0     -         LIBRARY_LOCATION_USER -
Arduino_HTS221    1.0.0     -         LIBRARY_LOCATION_USER -
ArduinoSound      0.2.1     -         LIBRARY_LOCATION_USER -
DigitLed72xx      0.0.6     -         LIBRARY_LOCATION_USER -
IoAbstraction     2.0.6     -         LIBRARY_LOCATION_USER -
IRremote          3.4.0     -         LIBRARY_LOCATION_USER -
LiquidCrystal_I2C 1.1.2     -         LIBRARY_LOCATION_USER -
LiquidCrystalIO   1.4.1     -         LIBRARY_LOCATION_USER -
RTClib            1.14.1    -         LIBRARY_LOCATION_USER -
SD                1.2.4     -         LIBRARY_LOCATION_USER -
Servo             1.1.8     -         LIBRARY_LOCATION_USER -
TaskManagerIO     1.3.0     -         LIBRARY_LOCATION_USER -
TinyWireM         1.1.0     -         LIBRARY_LOCATION_USER -
TM1637            1.2.0     -         LIBRARY_LOCATION_USER -
TM1637_Driver     2.1.1     -         LIBRARY_LOCATION_USER -
USBHID            0.63      -         LIBRARY_LOCATION_USER -
WiFiNINA          1.8.12    1.8.13    LIBRARY_LOCATION_USER Enables network connection (local and...

Arduino-cli shows only for WiFiNINA that a new version is available.

Run arduino-cli upgrade:

$ arduino-cli upgrade
Downloading WiFiNINA@1.8.13...
WiFiNINA@1.8.13 already downloaded
Installing WiFiNINA@1.8.13...
Replacing WiFiNINA@1.8.12 with WiFiNINA@1.8.13...
Installed WiFiNINA@1.8.13

Only WiFiNINA was upgraded

System:

$ uname -a
Linux raspberrypi 5.10.63-v7l+ #1459 SMP Wed Oct 6 16:41:57 BST 2021 armv7l GNU/Linux
$ arduino-cli version
arduino-cli alpha Version: 0.19.3 Commit: 12f1afc2 Date: 2021-10-12T10:15:19Z

Might be related to #932.

josefwegner commented 2 years ago

Also, I cannot upgrade the library by setting its name as a parameter in the lib upgrade command:

$ arduino-cli lib upgrade "Adafruit NeoPixel"
$ arduino-cli lib upgrade Adafruit\ NeoPixel
$ arduino-cli lib upgrade Adafruit_NeoPixel

All three commands do not have any effect.

nicholaswilde commented 2 years ago

A work around for me is to uninstall the library and the reinstall it.

arduino-cli lib uninstall "Adafruit SH110X"
arduino-cli lib install "Adafruit SH110X"

Ubuntu 20.04.4 arduino-cli Version: 0.21.1 Commit: 9fcbb392

stonehippo commented 2 years ago

I was running into this on multiple platforms (MacOS, debian/Raspberry Pi OS) so I decided to take a look. The issue is definitely in the resolution of existing installed package names versus the indexed package names. If a package contains spaces in its name field in library_index.json, it won't show up as upgradeable, since the installed name will typically include underscores in place of the spaces. This can be reproduced pretty easily. First, if we install a couple of libraries with old versions

$ arduino-cli lib install RTCZero@1.0.0 "Adafruit BMP280 Library"@1.0.0

then show the list:

$ arduino-cli lib list
Name                                             Installed Available Location              Description
Adafruit_BMP280_Library                          1.0.0     -         LIBRARY_LOCATION_USER -
RTCZero                                          1.0.0     1.6.0     LIBRARY_LOCATION_USER Allows to use the RTC functionalities...

Trying to upgrade RTCZero works fine:

$ arduino-cli lib upgrade RTCZero
Downloading RTCZero@1.6.0...
RTCZero@1.6.0 already downloaded
Installing RTCZero@1.6.0...
Replacing RTCZero@1.0.0 with RTCZero@1.6.0...
Installed RTCZero@1.6.0

Doing the same with the Adafruit BMP280 Library with any permutation fails to upgrade.

Digging around in the code a little, I think this is down to the implementation of FindIndexedLibrary in `arduino/libraries/librariesindex/index.go'. It currently looks like this:

func (idx *Index) FindIndexedLibrary(lib *libraries.Library) *Library {
    return idx.Libraries[lib.Name]
}

That isn't going to work, because the installed lib name will be something like "Adafruit_BMP280_Library" and the index name will be "Adafruit BMP380 Library". I tried a trivial fix:

func (idx *Index) FindIndexedLibrary(lib *libraries.Library) *Library {
    clean := strings.ReplaceAll(lib.Name, "_", " ")
    return idx.Libraries[clean]
}

Which ends up looking like this after a build:

Name                                             Installed Available Location              Description
Adafruit_BMP280_Library                          1.0.0     2.6.2     LIBRARY_LOCATION_USER Arduino library for BMP280 sensors.
RTCZero                                          1.0.0     1.6.0     LIBRARY_LOCATION_USER Allows to use the RTC functionalities...

The lib upgrade and lib upgrade [library name] commands also work correctly after this change.

@ubidefeo if this looks like a reasonable fix, I'll submit a PR.

ubidefeo commented 2 years ago

@stonehippo Let's run this through @cmaglie and see what he has to say