Closed mdeweerd closed 2 years ago
Ensure that the dates of the schematic/PCB are in line with the most recent change (latest git change for instance/local date).
Is this a git pull
run to get the last changes? Note that this is always done in CI/CD environments.
Generate a black&white schematic as well as color schematic - PDF and SVG versions;
Yes, you can generate it.
Update a text file describing the PCB technology (size, thickness, minimum track width, minimum clearance, minimum drill size, etc);
Automatically? Where do you get this info?
Generate PNG snapshots of the 3D PCB view:
Currently KiAuto doesn't generate it.
Keep a reference to the corresponding GIT hash, add/update a GIT tag;
Many users implement a tag replace, I would like to add such a feature, but I'm not sure about how to implement it.
Name the files or directory according to the board version;
KiBot can generate ZIP, RAR and tarballs with the output files. You can control various aspects (archive name, tree structure, etc.).
Copy the kicad_pcb file to maintain a copy of the PCB layout for that version in source format with the PCB files (usefull for tools such as Interactive BOM as well).
This looks like something you should do adding a tag to the repository.
Update a document (openoffice/libreoffice/word) with the PCB layers and generate a PDF from it. (Could be some '.md' or other file that pandoc understands well enough to generate a nice document).
Sound quite personal, but I'm curious about what information do you include there.
The BOM file would already be generated using KiCOST.
If you will be using KiBot I strongly recommend using it for the BoM, it uses KiCost engin, but also adds much more control about the output.
@set-soft Happy to discuss !
Ensure that the dates of the schematic/PCB are in line with the most recent change (latest git change for instance/local date).
Is this a
git pull
run to get the last changes? Note that this is always done in CI/CD environments.
I refer to the dates that appear on the schematic and on the PCB sheet - the ones that are visible when printed.
Currently they need a manual update. The goal is to make sure that they do not reflect the date of the previous version but of the most recent actual change or the date of release.
Some changes to the schematic do not justify an update of the date (I personnally add comments outside the border of the schematic with design condiderations, components, recommendations and constraints that do not appear on the printed schematic and that I may add given the experience of the actual PCBs).
Generate a black&white schematic as well as color schematic - PDF and SVG versions;
Yes, you can generate it.
Good, I'll tick the checkbox, and look for it when using KiBot ;-).
Update a text file describing the PCB technology (size, thickness, minimum track width, minimum clearance, minimum drill size, etc);
Automatically? Where do you get this info?
Generate PNG snapshots of the 3D PCB view:
Currently KiAuto doesn't generate it.
I suspected so ;-). So it's a wish.
Keep a reference to the corresponding GIT hash, add/update a GIT tag;
Many users implement a tag replace, I would like to add such a feature, but I'm not sure about how to implement it.
I suppose that you know how to replace a tag, so your question would be about methodology.
If I would implement a tag replace automatically, I would add a tag in place of the previous tab indicating it was replaced. For instance my TAG is "BOARDNAME_VERSION_B", and this tag already exists. Then I do the "release BOARDNAME_VERSION_B" command again. The script would look for "BOARDNAME_VERSION_B" in the tag list, add the "BOARDNAME_VERSION_B_REPLACED_20211028T101230Z" and then move the ""BOARDNAME_VERSION_B" to the current version.
I would possibly even add a tag "BOARDNAME_VERSION_B_20211028T101230Z" along with "BOARDNAME_VERSION_B" so that I know that the tag was placed as such on that date and time.
The git hash is something that could be added to the filename and/or a PDF/SVG/GERBER property. In C SW projects, I add the git hash as a constant in the compiled code and I add "M" when there are modified files (as SVN does).
Name the files or directory according to the board version;
KiBot can generate ZIP, RAR and tarballs with the output files. You can control various aspects (archive name, tree structure, etc.).
Ok, the idea is to have some kind of board version variable, used in appropriate locations.
Copy the kicad_pcb file to maintain a copy of the PCB layout for that version in source format with the PCB files (usefull for tools such as Interactive BOM as well).
This looks like something you should do adding a tag to the repository.
Git can make a tag for me, but it is practical to have the gerber file available without having access to git. A copy of the zipped production folder should suffice. I keep copies of built versions in subdirectories that are themselved in git. I can compare versions without checking them out specifically. Once they are built, they exist in the real world - easy access comes in handy from time to time.
It's also compatible with more generic archiving methods
Update a document (openoffice/libreoffice/word) with the PCB layers and generate a PDF from it. (Could be some '.md' or other file that pandoc understands well enough to generate a nice document).
Sound quite personal, but I'm curious about what information do you include there. Ok, Il try to make an example that I can share. Basically it becomes a single PDF:
- Front page, project name, with images of the 3D view on the front page,
- the BOM(s),
- instructions/limitations on the choice of the components, contacts for specific components,
- pages for each production layer with the image of the layer, for the back layer I also include a mirrored image next to the normal image (easier to compare with the real board) (so for copper, silk, fab, mask),
- pages indicating potential issues, how to detect them, how to avoid them and/or how to fix them;
- A pages with changes made to the project;
- A footer indicating the version, past versions, page number, total number of pages.
The BOM file would already be generated using KiCOST. If you will be using KiBot I strongly recommend using it for the BoM, it uses KiCost engin, but also adds much more control about the output.
Once I get in to using KiBot, I may be using it for the BoM, but I already have a script allowing me to generate the BoMs from the CLI for multiple variants. I copy it from one project to the next.
I do not know if KiBot can handle the generation of multiple BOMs, and I avoid regenerating the BOM when wrapping up as I would already have verified the BOM and I do not want to risk generating a BOM with changes that I would not have noticed.
Adding a few more:
Note: to renumber, I rely on BrianAtDocumentedDesigns/RenumKicadPCB
in ../RenumKiCadPCB
and have this script using Cygwin (I still have to type R to do the actual job) :
#!/bin/bash
# RENUMBER COMPONENTS
cat > RenumParameters.txt <<'EOF'
PCB file name=PROJECTNAME
Top Sort Direction=1
Bottom Sort Direction=5
Top Start Reference Designation=1
Bottom Start Reference Designation=101
Sort on Modules/Reference Designators=0
Sort Grid=1.000000
Log=0
EOF
#../RenumKiCadPCB/RenumKiCadPCB <<< "R"
../RenumKiCadPCB/RenumKiCadPCB
@mdeweerd You may find my GitHub workflows useful to look at. I've got date/version replacement happening on GitHub for example. It also generates complete zipped output and attaches it to GitHub releases.
See https://github.dev/neilenns/TBM930_De-ice_panel and the included CodeTour that walks through all of it.
@neilenns Thanks for the hint, I'll have a look at it in more detail later. I am currently configuring the project's yaml.
@set-soft
@mdeweerd I just finished setting up a GitHub repository template since I keep doing the same thing over and over again, and added a bunch more documentation.
@neilenns I made a reply yesterday, but my computer crashed - apparently it wasn't sent. I can't go through all that learning curve for now, but I'll look into it later. I've progressed quite a bit in the yaml configuration and I think the issues are mostly with KiBot (and used tools) itself.
@set-soft
- There is a global directory setting on the kibot command line, but the '%' references are not available there I suppose - the setting could be proposed (with inferior priority) in the yaml files.
It can be added, but: how do you plan to use it?
I currently set this in several targets:
- name: 'pdf_pcb_fab'
comment: 'Exports the PCB to the most common exhange format. Suitable for printing.'
type: 'pdf_pcb_print'
dir: "%f_%r_%d%v/PCBA"
And I think that 'dir' is designed there to be a subdirectory of the output directory which is currently only specifiable on the CLI.
The CLI does not know the version, etc, this is only known after reading the files. So I would prefer setting the directory like this:
globals:
dir: "%f_%r_%d%v"
- name: 'pdf_pcb_fab'
type: 'pdf_pcb_print'
dir: "PCBA"
The -d option on the command line could override this and the '%' expansion could be allowed there as well.
Also, when its in the yaml, we can just run 'kibot' without options and the outputs will be directed to the expected directory based on the PROJECT.kibot.yaml file(s) found.
Hi @mdeweerd !
I added a global dir
option but it works a little bit different. To get what you want you should do this:
globals:
dir: "%f_%r_%d%v/"
- name: 'pdf_pcb_fab'
type: 'pdf_pcb_print'
dir: "+PCBA"
Note the +
which means add to the global.
Also note that this doesn't affect the command line option -d
. The command line option remains unchanged.
@set-soft That perfect, more than I hoped for, certainly more generic.
I pulled the update.
Imported file (common to several projects):
kibot:
version: 1
global:
dir: "%f_%r_%d%v/"
outputs:
- name: "schematic_color"
type: "pdf_sch_print"
dir: "+PDF"
options:
# - dnf_filter:
output: "%f-%r%i%v.%x"
# - variant: ""
Local file (TEST.kibot.yaml):
kibot:
version: 1
import:
- common/common.yaml
and the directory generated was .PDF (with the dot).
- I am note sure that the globals from an import are included;
By default only outputs
are imported. You can import filters
and variants
, but not globals
- The default directory is '.', and it should become './' because of this improvement.
I agree this is more convenient. Done.
@set-soft Thank you for the update. Regarding the import of globals: that would be useful especially in this case where the definition is generic. I did not succeed in getting the expected directory - I'll investigate in more detail but in principal I have pulled the latest version and it's used in the docker image through a volume mount (which I put in place for debugging).
More than half of the tickboxes are checked now ;-).
I think that several items are not to be "natively" implemented in kibot, but would be manageable through custom outputs (#109) that allow us to get design information as parameters to the script (date, ...) and output parameters (directory, output filename, options) as well.
And there are those that I think are a task for kibot: B&W (monochrome) schematic, and updating date nand time (@neilenns does so in his flow, but I'ld prefer to have it in kibot),
Updating a template could be implemented by providing information (as JSON) the custom script, and would ideally get some information not currently available that would require extra scripts to be added to KiAuto (e.g., minimum drill, minimal clearance, track width, etc). So with custom scripts, this could be done in part already.
Generating a QRCode is yet another UI action, or calling the existing python script (KiCad/share/kicad/scripting/plugins/kicad_qrcode.py ) with the proper setup. This could be a custom script, or an addition to KiAuto & Kibot. Borderline really.
Renumbering references is something I do occasionnaly with BrianAtDocumentedDesigns/RenumKicadPCB (explained here. Could be handled through a custom output as well. But it could also in KiAuto and Kibot natively.
I prepared some prototype production files this WE. I managed to have all files generated automatically - there were some minor limitations/issues I had to work around (issues already opened and one fixed today).
I added monochrome schematic in KiAuto 1.5.12, please test it, was tested with some trivial example.
Monochrome schematic: tested in project & works! Box ticked! 👏 .
- Ensure that the dates of the schematic/PCB are in line with the most recent change (latest git change for instance/local date).
Can you elaborate it?
- Use the comments in the names (not available as % values currently)
Do you mean the comment lines from the title box?
Yes. The comments from the Page Settings. In Comment 4, I put the PCB reference. This can then be added automatically to the filename.
And later - with custom commands or filters - this could be used to fill in a text file with some of the PCB information.
Thanks ;-).
Yes. The comments from the Page Settings. In Comment 4, I put the PCB reference. This can then be added automatically to the filename.
Added
Added two new pre-flight options to do:
The replacements are constrained:
To reduce the changes of breaking the PCB/SCH. If you need to introduce these characters you must do it in the original file and perform small replacements.
They can replaced fixed text, or the output of a shell command. So you can customize the date origin and format and add things like the git hash.
@set-soft Ooooooh those are nice changes. I see the info in the docs, do you have an example of a file showing it in use? Also is this change available in a Docker tagged image somewhere?
Hi @neilenns !
@set-soft Ooooooh those are nice changes.
:-)
I see the info in the docs, do you have an example of a file showing it in use?
The PCB replacement test is pcb_replace_1.kibot.yaml and is applied to light_control.kicad_pcb (date and comment 4 of the title box are changed).
The SCH replacement test is sch_replace_1.kibot.yaml and is applied to test_v5.sch and its sub-sheets: deeper.sch and sub-sheet.sch (similar changes).
Also is this change available in a Docker tagged image somewhere?
You need the development tag: setsoft/kicad_auto:dev
Just run docker pull setsoft/kicad_auto:dev
and use this image.
Note this changes all the time, is the last code that passed all the regression tests.
- Copy the kicad_pcb file to maintain a copy of the PCB layout for that version in source format with the PCB files (usefull for tools such as Interactive BOM as well). I am not sure that we can copy files to the output directory using the project yaml (not the workflow).
Now you can manually add them to a compressed output. I know this isn't the ideal, but you can create an archive containing the files. The compress_sources_1.kibot.yaml is a very naive test (no lib or similar stuff).
[ ] Update a document (openoffice/libreoffice/word) with the PCB layers and generate a PDF from it. (Could be some '.md' or other file that pandoc understands well enough to generate a nice document).
Pandoc looks nice, but is a whale ... more than 155 MiB What is added to the layers? I mean: what's the added value here? Do you have an example?
Note: I tried with the README.md of this project and had some success for ODT and DOCX outputs. The PDF output needs LaTeX (probably over than 1 GiB) and fails with a "too deep" error. The RTF output generated something that Libreoffice can't interpret. And the others doesn't seem to be useful for this use.
I believe I promised somewhere to share the text file that I add to the PCB manufacturing files with regards to the parameters that could be extracted from the design files - for instance the size, but also minimum drill size, etc (not all can be extracted). The Class references the class as defined on the Eurocircuits site.
Example:
PCB Specifications:
Size:
- 168x38mm
Class: 3A
Track width:> = 0.250 mm
Insulation distance:> = 0.250 mm
Minimum drill size :> = 0.60 (finished metalized hole: 0.50mm)
Minimum slot width: 0.8mm
Ring collar: 0.2mm
Materials:
- FR4, 1.6mm
- HAL
- 2 layers
- 35um copper thickness
Solder mask:
- TOP / BOTTOM
- Black
Marking:
- TOP / BOTTOM screen printing
- Silk: White
Other markings:
- ROHS / UL / Date - Yes if available
Ok, I'm sure that the current report functionality isn't enough to cover every nice report you can imagine, but is a good starting point.
I was able to generate the following report that includes the schematic and top/bottom PCB layers.
The full report is actually a markdown file.
The above file was generated using pandoc. I first generated an ODT and then exported it to PDF (using LibreOffice).
@set-soft
Thanks for following up (I am not back yet to PCB design activities).
This is definitively very good for generating a "technical" datasheet of the manufacturing requirements for the PCB. Which are in turn needed to prepare an order and to make sure that the target goals are met.
I use pandoc regularly and you may find some inspiration in these two scripts I have in my bin directory (with some "lua" scripts that are used). One of them goes directly from Markdown to PDF. I am not using those two scripts very often but I just tested the PDF generation on Markdown document with images and it produced a good enough result.
I work on Windows with Cygwin, so there is some cygwin specific stuff that is easily converted to "*nix" - you probably just need to throw out the cygpath stuff. The parameter is the markdown document filename and the output is the same file with pdf extension instead of md.
(You can reuse those scripts I wrote, the lua's are probably not needed for this use - they are probably from https://gist.github.com/MyriaCore/75729707404cba1c0de89cc03b7a6adf#file-fix-links-lua and https://gist.github.com/lierdakil/00d8143465a488e0b854a3b4bf355cf6 ).
I didn't go down far enough in your sample to see that you've also added schematic and PCB drawings - very nice!
So an even better start to generate documentation !
I didn't go down far enough in your sample to see that you've also added schematic and PCB drawings - very nice!
So an even better start to generate documentation !
They are currently PDF files, but I plan to add PNG outputs to have better control on the quality.
I don't like using PNG outputs that much because they pixelize and make the document "heavier".
After adding rsvg-convert to my setup, I could generate the following PDF using the generated SVG files.
The viewBox of the generated SVG files needs to be cropped to the right size, but the quality is good.
I don't like using PNG outputs that much because they pixelize and make the document "heavier".
Did you try to generate a document using KiBot and pandoc? Or is just a custom flow. Using PDF (generated by KiCad) + pandoc (ODT output) the images has poor resolution. They are bitmaps at low DPIs. I'll see what is generated using SVGs, but using PDFs (what is currently available) isn't very good.
BTW: SVGs can be used for schematics, but not for PCB printed layers (only plotted)
Using the SVGs with Markdown and pandoc was a test to see if that flow was possible.
I do not know what is not possible for the PCB layers. They can be plotted (which is good for a page for each layer) to SVG, and they can be exported to SVG: page size down to the board (size) only and apparently also mirrored (which is nice for the back copper layer and silk):
I do not know what is not possible for the PCB layers
As I wrote: print is the problem, not plot. I'm adding support to KiAuto to print in SVG format, but using KiCad 6.0.2 I hit the following bug: https://gitlab.com/kicad/code/kicad/-/issues/11033 Exporting to SVG isn't currently supported by KiAuto, and I'm reluctant to add more of these "graphical" options. In fact this dialog changed in KiCad 6:
FYI, personnally I only used the "Plotted" SVGs but as far as I remember the issue is that they are not limited to the board size so I had to "crop" them using a manual method.
(I understand now that it's more a limitation of KiCAD with regards to the methods implemented in KiBOT).
The KiBOT bug is annoying of course, but probably an easy fix that will make it in the next version.
I'm adding a workaround for the SVG issue.
Excellent! 👍
- Renumber references (when it's the first version of the board).
I have some doubts about it: KiBot is all about automating routine tasks, and this seems to be a one time thing. I guess annotating small additions to the board could be interesting, but could go wrong and won't help in a CI/CD environment, unless you push the changes, something I don't like because you will push things that aren't reviewed by a human. Is this really some task for KiBot?
I agree that renumbering is not something that should be done in a CI/CD routine task, but that does not mean it doesn't call for automation (I have a script to call the renumbering tool).
Personnally, I see kibot as a "single stop shop". The kibot target does not have to be a default target but if I can do "kibot renumber" and it's done, then that's perfect. Not "many" scripts or manual operations (already greatly reduced bit kibot), but "only" kibot.
It"s not a "one time thing" because when you think you're done, there's always something that needs to be added or (re)moved. Having all the configuration settings in a single file is great (I use a yaml configuration that I share accross designs through git submodule).
Doing this between two official releases for a few minor changes is not a good idea of course, but between prototypes and before the board is "final", I do repeat this.
Regarding pushing (git) things: in the end a human is responsible for it. You still have to review things. Automation can go wrong for plenty of reasons (not only SW bugs). BTW, that's where diffing is interesting. When I make a new release I overlay the new version with the previous one, I carefully check the BOMs manually (using tools to help of course).
So it's on my wishlist ;-) and it will be something that I will always have some kind of automation for - I often say that if you need to do something once, you often have to do it twice or more - so I start scripting from the first try.
(I have a script to call the renumbering tool).
The problem with this tool is that this is a Windows + GUI tool, not even compilable using free software (I mean really free, not gratis) and that won't work for the currently stable KiCad (6.x). So we need a replacement.
I implemented the power annotation, but I'm not sure about a full annotation.
I am using a CLI version: https://github.com/mdeweerd/RenumKiCadPCB
And this is my script (unfortunately I still have to type the R to run) and I did not make the projectname variable.
#!/bin/bash
# RENUMBER COMPONENTS
cat > RenumParameters.txt <<'EOF'
PCB file name=PROJECTNAME
Top Sort Direction=1
Bottom Sort Direction=5
Top Start Reference Designation=1
Bottom Start Reference Designation=101
Sort on Modules/Reference Designators=0
Sort Grid=1.000000
Log=0
EOF
#../RenumKiCadPCB/RenumKiCadPCB <
My sample script in the repository is better than what I have in my project ;-) - I made the project name variable: https://github.com/mdeweerd/RenumKiCadPCB/blob/main/samplescript.sh
My sample script in the repository is better than what I have in my project ;-) - I made the project name variable: https://github.com/mdeweerd/RenumKiCadPCB/blob/main/samplescript.sh
I took a look to this tool. Is interesting, but isn't what most people like. I now realize it numbers the components according to their PCB position. This is something you don't do on a medium (or big) project, because you work plenty of time on the schematic and then do the PCB. The numbers assigned during schematic work won't change, and you don't want them to be changed because you already used them on your design notes (and shared their reference with other people in the team).
I agree that looking at a PCB where components are numbered according to their physical position is nice, but I see it practical only for small projects.
BTW tools like BoardView are designed to help with the "mess" of physical positions.
Also: this tool is for KiCad 5 and KiCad 6 schematic changed a lot.
As I already have some annotation code, and the Python API allows a cleaner access to the PCB (this tool relies on heuristics to patch the netlist of the PCB and collect modules information) I'll play with the concept.
I do not know what most people like, but it's appreciated in the field whan you are looking for R68 on the PCB, and more when the board drawing or tools are not available on location.
I also find it useful in the lab.
I know of one company that refused to add component values on the schematic claiming that they are already mentioned in the BOM and always subject to change.
That's just to show that there's always a "good" reason to do one way or the other.
Every team/person will work differently, but somebody made this tool, and I am a user of the tool.
KiCad itself has an integrated renumbering feature, but it renumbers based on the position in the schematic and to ensure consistency between the schematic and the PCB it (can) use(s) another key value ( you can choose how to update the PCB from the schematic).
In the beginning of my career and a project that was composed of one main board that was about "A4" size, with several daughter boards covering about as much, the team responsible for drawing the PCB renumbered the components without asking the designer's opinions.
In a big project I find that you're more often validating smaller parts of the circuits and porting several "subcircuits" to the next project evolution (which generally implies a new numbering.
That itself makes whatever numbering you have in a notebook "irrevelant" in terms of the new project, but still relevant in the note because you have to keep a copy of the schematic at that time with the corresponding numbering.
But I hear that you're going to play with the concept and appreciate your openness on the topic.
In a big project I find that you're more often validating smaller parts of the circuits and porting several "subcircuits" to the next project evolution (which generally implies a new numbering.
This is why we use individual numbering for each page in the schematic. Each page starts its numbering from PAGE*100. And is also useful when you know that R101 and C201 doesn't belong to the same function and that you'll find what's R101 in page 1 and C201 in page 2 ;-)
And yes, I'm open, in fact I already implemented the PCB renumbering code. Needs a lot of testing (8 possible orders ...) and I need to feed the changes back to the schematic (trivial for KiCad 6).
There are advantages in each numbering method, but I personally find that we need to go from schematic to board more often than from the board to the schematic.
Ok, I added some implementation. Has most of the options found in the original tool. I did some tests, but will need adjusts.
Ok, I think is time to close this discussion and test the new goodies.
Very soon now, thanks. I'll try to share my setup more officially than through a project used for debugging.
I am just about to look into KiBot, AFAICS, KiBot can help with most things, but I am not sure that it generates B/W and color schematics, or generates the PNG files from the 3D viewer.
So just I am sharing what I like to do when wrapping up a KiCAD project, which may give some ideas for future updates. I'll try to tick the boxes that are already possible.
Edit: The "3D Viewer" is now intergrated into pcbnew_do to generate captures from it allowing all of the following.
I am not sure that we can copy files to the output directory using the project yaml (not the workflow).
The BOM file would already be generated using KiCOST.
(Adding a few more:)
I run most things through native scripts, but this project, once I get it working, may help getting the 3D images and STEP file through automation. Several other steps are automated using local scripts started under cygwin. Edit: -> I am moving this to KiBot using the prebuilt docker files.