INTI-CMNB / KiBot

KiCad automation utility
GNU Affero General Public License v3.0
571 stars 67 forks source link

[FEATURE] Update docs with KiCad 8 preflight erc/drc and filters. #639

Closed oliv3r closed 2 months ago

oliv3r commented 4 months ago

Looking at preflight.rst, we see a lot of new details with regards to KiCad8 missing, only some old 'run_drc' is listed or kicad5 stuff.

Also when generating the example template, we only get a very plain(, non functional example?).

  filters:
    - filter: 'Filter description'
      error: '10'
      regex: 'Regular expression to match'

Reason for my asking, is I can't figure out to ignore the following:

preflight:
  drc:
    enabled: true
    filters:
      - change_to: 'ignore'
        filter: 'KiCad bug #6918, self-contained project fails with 3D models'
        error: 'lib_footprint_mismatch'
      - change_to: 'ignore'
        filter: 'Earth ground is connected via connector shell, drc-flood-fills resets this'
        error: 'unconnected_items'
        regex: '[Earth]'

where kibot produces

- Running the DRC
ERROR:12 DRC errors detected (kibot.pre_drc - pre_drc.py:150)
ERROR:(unconnected_items) Missing connection between items
    @(131.6525 mm, 69.38 mm): Pad 2 [Earth] of R3 on F.Cu
    @(131.5 mm, 67.7425 mm): Track [Earth] on F.Cu, length 0.1945 mm
    Check: Unconnected items (kibot.pre_drc - pre_drc.py:165)
ERROR:(unconnected_items) Missing connection between items
    @(123.3475 mm, 69.38 mm): Pad 2 [Earth] of R1 on F.Cu
    @(123.5 mm, 67.7425 mm): Track [Earth] on F.Cu, length 0.1945 mm
    Check: Unconnected items (kibot.pre_drc - pre_drc.py:165)

These are 'earth' pads which connect to the shell of a connector, which is where these parts are 'connected', so I want to ignore these. I could exclude this within kicad, but because these are on ground-planes, kicad keeps 'resetting' these ...

@@ -67,13 +67,13 @@
         "unconnected_items|101000000|71380000|1e77a55c-6b27-407a-a9f1-3fecab808215|53be1935-eda4-4c71-8813-4adc466f0bbb",
         "unconnected_items|109000000|65380000|1dd9e8d9-7362-4bbb-8c13-5b72ce1b59c0|14c2e064-7c91-436f-8d68-b1e822b3b3ef",
         "unconnected_items|109000000|71380000|763c239b-beb3-4f8e-be2f-e9c1d0044f29|1dd9e8d9-7362-4bbb-8c13-5b72ce1b59c0",
-        "unconnected_items|123347500|69380000|57ec9a78-0d72-49b7-820e-23a06a4a7238|2d7e7e78-f86c-4a27-8953-a0b1d4e71ecf",
+        "unconnected_items|123347500|69380000|ad2be1e7-3aa9-4a4b-b954-55d237cea06a|2d7e7e78-f86c-4a27-8953-a0b1d4e71ecf",
         "unconnected_items|131652500|69380000|32f203c6-5269-4730-ba4f-9043c077ed7b|90610fd6-0ebf-4c59-8f63-177dd5b778b3",
         "unconnected_items|131652500|69380000|32f203c6-5269-4730-ba4f-9043c077ed7b|ba9b2ccf-dc68-45f6-9886-d64534cbd252",
         "unconnected_items|146000000|71380000|ba9b2ccf-dc68-45f6-9886-d64534cbd252|d1321477-53b0-460d-9912-d4ad2b202e67",
         "unconnected_items|64000000|65380000|bf013c16-da36-4d65-961e-faa0aaab3955|1d25011d-987c-4a3f-a868-d3060282eeb7",
         "unconnected_items|64000000|71380000|1d25011d-987c-4a3f-a868-d3060282eeb7|10deb9e3-8a44-4960-9a21-d2fa8d6d4774",
-        "unconnected_items|78347500|69380000|675f5d02-0fb1-497f-9c19-57727ccb9b9b|87ba23fe-9192-4f28-8f72-85019276b219",
+        "unconnected_items|78347500|69380000|675f5d02-0fb1-497f-9c19-57727ccb9b9b|ddfdd648-4231-427a-be26-d42a28f4eab3",
         "unconnected_items|86637500|67880000|6fb52b2e-3775-4405-86b6-beb00c757057|0a12e8cf-618a-408b-806d-1febf15a1780",
         "unconnected_items|86750000|67880000|38d0187a-9f6c-4406-9985-2439ae96b7b9|53be1935-eda4-4c71-8813-4adc466f0bbb"
       ],
set-soft commented 4 months ago

Looking at preflight.rst, we see a lot of new details with regards to KiCad8 missing, only some old 'run_drc' is listed or kicad5 stuff.

Looking at the master I can see drc and erc documented. I agree that run_drc should also say its deprecated, as run_erc does. But drc clearly states this is the one for KiCad 8.

Also when generating the example template, we only get a very plain(, non functional example?).

  filters:
    - filter: 'Filter description'
      error: '10'
      regex: 'Regular expression to match'

I guess you mean the one generated with kibot --example, this example clearly states:

# ATTENTION! THIS ISN'T A FULLY FUNCTIONAL EXAMPLE.
# You should take portions of this example and edit the options to make
# them suitable for your use.
# This file is useful to know all the available options.

You'll get a better example using --quick-start, as the docs suggest.

Reason for my asking, is I can't figure out to ignore the following:

preflight:
  drc:
    enabled: true
    filters:
      - change_to: 'ignore'
        filter: 'KiCad bug #6918, self-contained project fails with 3D models'
        error: 'lib_footprint_mismatch'
      - change_to: 'ignore'
        filter: 'Earth ground is connected via connector shell, drc-flood-fills resets this'
        error: 'unconnected_items'
        regex: '[Earth]'

where kibot produces

- Running the DRC
ERROR:12 DRC errors detected (kibot.pre_drc - pre_drc.py:150)
ERROR:(unconnected_items) Missing connection between items
    @(131.6525 mm, 69.38 mm): Pad 2 [Earth] of R3 on F.Cu
    @(131.5 mm, 67.7425 mm): Track [Earth] on F.Cu, length 0.1945 mm
    Check: Unconnected items (kibot.pre_drc - pre_drc.py:165)
ERROR:(unconnected_items) Missing connection between items
    @(123.3475 mm, 69.38 mm): Pad 2 [Earth] of R1 on F.Cu
    @(123.5 mm, 67.7425 mm): Track [Earth] on F.Cu, length 0.1945 mm
    Check: Unconnected items (kibot.pre_drc - pre_drc.py:165)

Please provide a simple example. The filter regex doesn't look right, but should be filtered anyways. You should read something about regular expressions, [ is a special character.

oliv3r commented 4 months ago

Looking at preflight.rst, we see a lot of new details with regards to KiCad8 missing, only some old 'run_drc' is listed or kicad5 stuff.

Looking at the master I can see drc and erc documented. I agree that _rundrc should also say its deprecated, as _runerc does. But drc clearly states this is the one for KiCad 8.

I appologize, I wasn't clear enough. I meant the examples specifically. There's only run_drc examples (as quoted below), but there's no new kicad8 drc example.

Also when generating the example template, we only get a very plain(, non functional example?).

  filters:
    - filter: 'Filter description'
      error: '10'
      regex: 'Regular expression to match'

I guess you mean the one generated with kibot --example, this example clearly states:

# ATTENTION! THIS ISN'T A FULLY FUNCTIONAL EXAMPLE.
# You should take portions of this example and edit the options to make
# them suitable for your use.
# This file is useful to know all the available options.

You'll get a better example using --quick-start, as the docs suggest.

I did, and I know it's just that, a not fully example, but with error: '10' it hints to beign a run_drc example. But it also means I was not able to extract what I should be using.

Reason for my asking, is I can't figure out to ignore the following:

preflight:
  drc:
    enabled: true
    filters:
      - change_to: 'ignore'
        filter: 'KiCad bug #6918, self-contained project fails with 3D models'
        error: 'lib_footprint_mismatch'
      - change_to: 'ignore'
        filter: 'Earth ground is connected via connector shell, drc-flood-fills resets this'
        error: 'unconnected_items'
        regex: '[Earth]'

where kibot produces

- Running the DRC
ERROR:12 DRC errors detected (kibot.pre_drc - pre_drc.py:150)
ERROR:(unconnected_items) Missing connection between items
    @(131.6525 mm, 69.38 mm): Pad 2 [Earth] of R3 on F.Cu
    @(131.5 mm, 67.7425 mm): Track [Earth] on F.Cu, length 0.1945 mm
    Check: Unconnected items (kibot.pre_drc - pre_drc.py:165)
ERROR:(unconnected_items) Missing connection between items
    @(123.3475 mm, 69.38 mm): Pad 2 [Earth] of R1 on F.Cu
    @(123.5 mm, 67.7425 mm): Track [Earth] on F.Cu, length 0.1945 mm
    Check: Unconnected items (kibot.pre_drc - pre_drc.py:165)

Please provide a simple example. The filter regex doesn't look right, but should be filtered anyways. You should read something about regular expressions, [ is a special character.

Ah, sorry, that was my copy/paste in my feeble attempts to see what would work. I initially had just Earth which is fine, but that wasn't triggered uppon. I am fully aware of the special meaning of the [ character, and know it probably would need to be escaped, but I never got that far. So I appologize for the incorrect example.

Here's a an example with everything: https://gitlab.com/olliver/lapod/-/jobs/7268429127 and yes, I am fully aware it's a complex pipeline example; but the behavior can be replicated after cloning of the repo by doing

docker run --rm -it -v ./:/w -w /w ghcr.io/inti-cmnb/kicad8_auto_full:1.7.0 kibot -c breakout_hdmi/.kibot/kici_drc.kibot.yaml -e breakout_hdmi/breakout_hdmi.kicad_sch -b breakout_hdmi/breakout_hdmi.kicad_pcb
set-soft commented 4 months ago

Here's a an example with everything: https://gitlab.com/olliver/lapod/-/jobs/7268429127 and yes, I am fully aware it's a complex pipeline example; but the behavior can be replicated after cloning of the repo by doing

I tried it and is using run_drc, not drc. Changing it to drc I got:

- Running the DRC
WARNING:(W144) Using filters from the `filters` preflight, move them to `drc` (kibot.pre_drc - pre_drc.py:39)
WARNING:(W145) 8 DRC warnings detected (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(122.3875 mm, 67.88 mm): Footprint C1
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(77.5 mm, 67.88 mm): Footprint C2
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(132.5 mm, 67.88 mm): Footprint C3
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(87.6125 mm, 67.88 mm): Footprint C4
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(122.75 mm, 69.38 mm): Footprint R1
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(77.6525 mm, 69.38 mm): Footprint R2
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(132.25 mm, 69.38 mm): Footprint R3
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(87.25 mm, 69.38 mm): Footprint R4
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
Found 10 unique warning/s (10 total)

I also tried with a docker image, and I can't reproduce it.

oliv3r commented 4 months ago

Here's a an example with everything: https://gitlab.com/olliver/lapod/-/jobs/7268429127 and yes, I am fully aware it's a complex pipeline example; but the behavior can be replicated after cloning of the repo by doing

I tried it and is using run_drc, not drc. Changing it to drc I got:

- Running the DRC
WARNING:(W144) Using filters from the `filters` preflight, move them to `drc` (kibot.pre_drc - pre_drc.py:39)
WARNING:(W145) 8 DRC warnings detected (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(122.3875 mm, 67.88 mm): Footprint C1
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(77.5 mm, 67.88 mm): Footprint C2
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(132.5 mm, 67.88 mm): Footprint C3
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(87.6125 mm, 67.88 mm): Footprint C4
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(122.75 mm, 69.38 mm): Footprint R1
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(77.6525 mm, 69.38 mm): Footprint R2
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(132.25 mm, 69.38 mm): Footprint R3
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
WARNING:(W145) (footprint_symbol_mismatch) Footprint attributes don't match symbol: 'Do not populate' settings differ.
    @(87.25 mm, 69.38 mm): Footprint R4
    Check: Schematic parity (kibot.pre_drc - pre_drc.py:144)
Found 10 unique warning/s (10 total)

I also tried with a docker image, and I can't reproduce it.

which is totally my fault for not mentioning that the kicad8 work is happening on the branch upgrade/kicad_8, which i admit is not completely obvious from the job log I think. The master stuff still is based on the old kicad with indeed the legacy run_drc. I couldn't merge it to master yet due to failures :p

Though probably a good idea would be to add a warning when using run_?rc on the kicad8 container/app?

set-soft commented 3 months ago

Hi @oliv3r !

I did a checkout for the branch you mentioned, and updated the .kici submodule. Then I got errors about unconnected items, but this was because it didn't have a filter, after adding the filter:

diff --git a/kibot_jobs/kici_drc.kibot.yaml b/kibot_jobs/kici_drc.kibot.yaml
index f0f7eca..79df495 100644
--- a/kibot_jobs/kici_drc.kibot.yaml
+++ b/kibot_jobs/kici_drc.kibot.yaml
@@ -11,3 +11,12 @@ preflight:
     enabled: true
     format: 'CSV,HTML,JSON,RPT'
     warnings_as_errors: true
+    filters:
+      - change_to: 'ignore'
+        filter: 'KiCad bug #6918, self-contained project fails with 3D models'
+        error: 'lib_footprint_mismatch'
+      - change_to: 'ignore'
+        filter: 'Earth ground is connected via connector shell, drc-flood-fills resets this'
+        error: 'unconnected_items'
+        regex: '[Earth]'
+

I get no errors

set-soft commented 3 months ago

Though probably a good idea would be to add a warning when using run_?rc on the kicad8 container/app?

The warning is there!

$ kibot -c tests/yaml_samples/drc.kibot.yaml -b tests/board_samples/kicad_8/light_control.kicad_pcb  -d pp
WARNING:(W143) For KiCad 8 use the `drc` preflight instead of `run_drc` (kibot.pre_run_drc - pre_run_drc.py:86)

It was added 3 months ago: 7baff559412e5ba8134c96549eeb1c6ed7ee458c

set-soft commented 3 months ago

The above patch should update the filters explanation

oliv3r commented 3 months ago

Maybe this is because i'm using v1.7.0? I'll recheck with a fresh-checkout ...

set-soft commented 3 months ago

Maybe this is because i'm using v1.7.0? I'll recheck with a fresh-checkout ...

The 1.7.0 release includes the warning (W143)