[FEATURE] Refresh kipart libraries and footprint from database

[turdusmerula]

Add a way to build kicad library folders from models and footprints stored in kipartman.

[auphofBSF]

This can work both ways, The Kipartman can extract details from the Kicad Libraries (contained in LIB,DCM and .PRETTY, .KICAD_MOD) . I think as a start we should move footprint and model attributes to being a tab called ECADdata on the part view. Basically to come up with this model of an interface

I have created a quick branch and created ecaddata classes as copies of part_parameters, the edit_ecaddata form has yet to be modified to the above,(Currently just redisplays parameters, but should be able to use all the great work in kipartbase and kipartman for footprint and model lookup at present I have only published this branch to my Fork FEATURE17W48-ECADtab to not pollute yours.

1 additional and key [USECASE] here is the extraction and linking of kipartman - parts to kicad-lib, symbols and footprints if they exist. If they don't exist or need customization you use your great lookup on SNAP-EDA. The KiCad-Lib interface would be bidirectional. Updating the/our organization custom libraries located at paths {KIORG_SYMBOL_DIR} and {KIORGMOD} and {KIORG3DDMOD} as apposed to the KISYSMOD and KISYSD3DMOD and KICAD_SYMBOL_DIR

As mentioned yesterday I am tidying up some drawings and sketches I have of what my ideal component management workflow/interfaces would look like under Kicad. linking in the organizations, inventory/MRP together with engineering data, and automating the workflow in ECAD. What you have done here in Kipartman is great step to making this possible. I can see it is is now quite achievable. I am trying to get these further thoughts visible later today.

[turdusmerula]

I'm starting to understand and I'll take the time to take a closer look, I like the idea.

Don't lose sight of the future multi-user purpose of kipartman, this may have an impact on the design of this feature. Kipartbase and kipartman won't necessarily be on the same host in multiuser purposes. We have to think on what will be stored on the kipartbase side and what will be on kipartman side interacting with user.

[auphofBSF]

Absolutely, I believe that is one of the strengths of what you have here with kipartman, that there is a robust and multiuser (tobe) backend and yet a UI that can manage some complex activities and even extensible by a fairly green python hacker like myself.

I have tidied up my ideas around the workflow at schematic capture. I had encapsulated this in the notion of a suite of tools I called KiCADddm (Design Data Management) I have been searching for existing product that fits or does this. Kipartman I believe is the closest thing I have seen to enable a large part of Automating some of the workflow. I have opened my Github Repository https://github.com/auphofBSF/KiCadDDM for discussion purposes.

Take a look at (https://github.com/auphofBSF/KiCadDDM/blob/master/StateofKiCADddm201711.md) this attempts to describe and show what I am looking for at schematic capture.

[turdusmerula]

One implementation detail: As you could see for octopart / snapeda I was forced to maintain a mapping by hand to access the data-model, this is not that handy but it works without so much performance gap. For the internal data-model everything is managed by swagger, the model is described in the yaml files and generated with the swagger/generate.sh script (that will probably only run on linux). This script is here also to patch the generated result as I encountered some issues with recursive imports and packages naming collisions. Swagger also generates both server and client api for the REST protocol and does it very nicely.

To get back on the FEATURE, I'm starting to study it. When I opened this feature I was thinking of using kipartbase to store every footprint / models and kicad elements in kipartman. I was searching how to provide a way to allow user to synchronize his local kicad libraries from kiparbase. What is interesting me in what you describe is that you go further and propose to have a bidirectional link, what in the end is equivalent to superseding the kicad internal part management. Looks like a huge work that will require a closer integration with kicad API :)

[auphofBSF]

Regarding the Swagger generate script, I think that is fine that just runs on linux. That can probably even work from the bash shell on windows. I use it extensively. So I will try it. It is great the model is maintained in this way with automation of the API generation.

[auphofBSF]

Regarding the 2 repositorues , KipartBase and Kicad Libraries.
The kipartman/base generated footprint and symbol libraries is certainly needed to maintain organizational approved components. Any modification of symbol or footprint will need to be pushed back in a managed way into KipartBase with hopefully some GIT style version control. Kipartman will maintain more organizational data on components than what Kicad Lib will have. So we can only seed this data held in KipartBase from sources 1) KicadLib 2) SnapEda 3) Octopart 4) other imports.
Although all interfaces are fraught with complexity, this one can be kept quite strict, and largely based around limited import from KicadLib , symbol, footprint, and although other meta data is being stored, if these map across easily then all them as input into Kipart component generation. We might be able to do it without or with limited Kicad API. I would favour trying to port as much as possible to python classes.

[auphofBSF]

I am now successfully populating kicad fields from kipartman data in realtime This I believe is a Feature separate but related to this [FEATURE] #4. I will open a new [FEATURE] and document it there #18

[auphofBSF]

Have had a little difficulty installing cairo on MSW. I have used the conda cairo package as pip install pycairowas failing on compilation. Need to look what distributables are required to make it work in a non conda/anacoda environment on MSW. suggest I raise this as a separate issue/bug.

Success however was had getting working execution of kipartman, conda install -c anaconda pycairo this installs pycairo==1.13.3 a liitle behind 1.15.4 on github

TODO: not yet tested for rendering of mod, lib, pretty

[turdusmerula]

I hope there will be a solution for that or packaging the next version may be complicated. We will work on this separately when all other things will be ready. I have some test code I will push to test on windows.