[BUG] KiBot can't find libraries

[flaviut]

Describe the bug

When I run KiBot with my Kicad 6.0.9 installation, it has trouble finding the various system libraries. This is because the environment variables like KICAD6_SYMBOL_DIR are not included in the .config/kicad/6.0/kicad_common.json file for some reason. Manually adding them in works, but then Kicad removes them again when the settings are saved.

Not sure how this can be fixed--KiCad uses a compile-time configured path for symbol libraries: 1, 2.

Log output:

- 'Generic Bill of Materials in CSV format' (generic_bom_csv) [bom]
DEBUG:Output destination: /home/user/tmp/AWG/Assembly (kibot - kiplot.py:361)
DEBUG:Filters reset (kibot - fil_base.py:161)
DEBUG:Applying filter `_mechanical` to exclude (kibot - fil_base.py:154)
DEBUG:Applying filter `_kibom_dnf_Config` to fitted (kibot - fil_base.py:171)
DEBUG:Applying filter `_kibom_dnc_Config` to fixed (kibot - fil_base.py:181)
DEBUG:Applying KiBoM style variants `default` (kibot - var_kibom.py:106)
DEBUG:Creating internal filter: {'name': '_expand_text_vars', 'type': 'expand_text_vars', 'comment': 'Internal default text variables expander'} (kibot - fil_base.py:251)
DEBUG:Applying transform filter `_expand_text_vars` (kibot - fil_base.py:140)
DEBUG:Reading KiCad config from `/home/user/etc/.config/kicad/6.0/kicad_common.json` (kibot - config.py:411)
WARNING:(W010) Unable to find KiCad libraries (kibot - config.py:403)
DEBUG:Using KICAD_TEMPLATE_DIR="/usr/share/kicad/template" (from KiCad config) (kibot - config.py:362)
DEBUG:Using KICAD_USER_TEMPLATE_DIR="/home/user/kicad/template" (from KiCad config) (kibot - config.py:362)
WARNING:(W010) Unable to find KiCad footprints (kibot - config.py:403)
WARNING:(W010) Unable to find KiCad 3D models (kibot - config.py:403)
DEBUG:Using KICAD6_3RD_PARTY="/home/user/etc/.local/share/kicad/6.0/3rdparty" (guessed) (kibot - config.py:386)
DEBUG:Exporting KIPRJMOD="/home/user/tmp/AWG" (kibot - config.py:431)
DEBUG:Exporting KIGITHUB="https://github.com/KiCad" (kibot - config.py:431)
DEBUG:Loading symbols lib table `/home/user/etc/.config/kicad/6.0/sym-lib-table` (kibot - config.py:436)
ERROR:Unable to expand `KICAD6_SYMBOL_DIR` in `${KICAD6_SYMBOL_DIR}/4xxx.kicad_sym` (kibot - config.py:102)
DEBUG:Loading symbols lib table `/home/user/tmp/AWG/sym-lib-table` (kibot - config.py:436)
DEBUG:Loading 3D aliases from /home/user/etc/.config/kicad/6.0/3d/3Dresolver.cfg (kibot - config.py:491)
DEBUG:Finished loading 3D aliases (kibot - config.py:506)
DEBUG:Decimal point `.` (kibot - units.py:124)
DEBUG:Using auxiliary origin: x=0 y=0 (kibot - bom.py:481)
DEBUG:Saving BOM File: /home/user/tmp/AWG/Assembly/awg-bom.csv (kibot - bom.py:552)
DEBUG:CSV Output -> /home/user/tmp/AWG/Assembly/awg-bom.csv (kibot - bom_writer.py:50)

.config/kicad/6.0/kicad_common.json:

  "environment": {
    "show_warning_dialog": true,
    "vars": {
      "KICAD_TEMPLATE_DIR": "/usr/share/kicad/template",
      "KICAD_USER_TEMPLATE_DIR": "/home/user/kicad/template",
      "KIGITHUB": "https://github.com/KiCad"
    }
  },

Environment (please complete the following information): Where are you running KiBot:

• Using a desktop system? Which OS, distro and version? Arch Linux, kicad 6.0.9

[set-soft]

Hi @flaviut !

WARNING:(W010) Unable to find KiCad libraries (kibot - config.py:403)

Where are your libs installed? Aren't they at /usr/share/kicad/symbols/?

[flaviut]

They are!

$ ls /usr/share/kicad/symbols/
4xxx_IEEE.kicad_sym                      MCU_Microchip_SAMV.kicad_sym
4xxx.kicad_sym                           MCU_Module.kicad_sym
74xGxx.kicad_sym                         MCU_Nordic.kicad_sym
74xx_IEEE.kicad_sym                      MCU_NXP_ColdFire.kicad_sym
74xx.kicad_sym                           MCU_NXP_HC11.kicad_sym
Amplifier_Audio.kicad_sym                MCU_NXP_HC12.kicad_sym

[flaviut]

Looking at https://github.com/INTI-CMNB/KiBot/blob/1248e92b9556cbcaf82c3a36577bc98b0c9dc61f/kibot/kicad/config.py#L180, it seems like the autodetection only works for KICAD_SYMBOL_DIR rather than the KICAD6_SYMBOL_DIR involved here?

[set-soft]

Hi @flaviut ! Which version are you running? Try upgrading to 1.4.0

[flaviut]

ah, I see. dev is actually the main branch here, not master. If you'd like, you can adjust things on github so that dev is shown by default, and so that the github code search uses dev:

But I am on 1.4.0, freshly installed from pip about a week ago:

$ kibot -V
KiBot 1.4.0 - Copyright 2018-2022, Salvador E. Tropea/INTI/John Beard - License: GPL v3+

[set-soft]

Hi @flaviut ! The master is currently the last release, and dev is the WIP code. The current code doesn't use the line you mention. Can you try running the following Python script?

import os
import platform
import sysconfig

system = platform.system()
share = os.path.join('share', 'kicad', 'symbols')
if system == 'Linux':
    scheme_names = sysconfig.get_scheme_names()
    # Try in local dir
    if 'posix_user' in scheme_names:
        dir = os.path.join(sysconfig.get_path('data', 'posix_user'), share)
        print('Try '+dir)
        if os.path.isdir(dir):
            print(dir+' Ok')
            exit(0)
    # Try at system level
    if 'posix_prefix' in scheme_names:
        dir = os.path.join(sysconfig.get_path('data', 'posix_prefix'), share)
        print('Try '+dir)
        if os.path.isdir(dir):
            print(dir+' Ok')
            exit(0)
else:
    print("Not Linux")

[flaviut]

The master is currently the last release, and dev is the WIP code.

ok, I understand now.

Can you try running the following Python script?

Try /home/user/etc/.local/share/kicad/symbols
Try /home/user/tmp/AWG/venv/share/kicad/symbols

looks like it's trying to look for symbols in my virtualenv. doesn't seem like sysconfig works very well in this situation:

>>> sysconfig.get_paths()
{'stdlib': '/usr/lib/python3.10', 'platstdlib': '/home/user/tmp/AWG/venv/lib/python3.10', 'purelib': '/home/user/tmp/AWG/venv/lib/python3.10/site-packages', 'platlib': '/home/user/tmp/AWG/venv/lib/python3.10/site-packages', 'include': '/usr/include/python3.10', 'platinclude': '/usr/include/python3.10', 'scripts': '/home/user/tmp/AWG/venv/bin', 'data': '/home/user/tmp/AWG/venv'}

[set-soft]

Hi @flaviut ! Why do you need a virtualenv? I never use it. It looks like the virtualenv is preventing KiBot from finding vital information about your system. You could symlink the KiCad resources dir in your virtualenv (ln -s /usr/share/kicad /home/user/tmp/AWG/venv/share/kicad) Or you can just define the variables in your environment using export. You can use your ~/.bashrc file to make it persistent.

[flaviut]

Why do you need a virtualenv? I never use it.

It's good to have the correct version of kibot for each project. But now that you mention it, it doesn't make much sense since kicad is still from the system's packages.

Or you can just define the variables in your environment using export. You can use your ~/.bashrc file to make it persistent.

Thank you for the really good advice--these are great workarounds.

[set-soft]

Hi @flaviut ! Note that we currently have some docker images with specific versions of KiBot and KiCad. Currently GitHub is giving us enough space to host them, not sure how much images we will be able to keep. But if you really needed reproducibility you can use these images.