kikit stalls from plugin, but fast from CLI

gwideman commented 1 month ago

Prerequisites

[X] I have read FAQ
[X] I have searched existing issues (including closed ones)
[X] I use KiKit at least version 1.5.1 (older version are not supported)

KiKit version

kikit, version 1.5.1

KiCAD version

8.0.3

Operating system

Windows 10 Pro 10.0.19045

Description

This report may have insufficient detail to spot a cause, but I'm hoping for suggestions on how to help narrow down that cause.

Symptom: A panelization job, as specified in a json file, runs in seconds from the command line, but when launched from the Kicad PCB kikit plugin UI "Panelize" button, the kikit process stalls for several minutes.

Sample stall time: For a 35x35mm board with 5 SMD ICs and a few dozen other SMD parts, to be panelized 3x4 boards, from CLI it runs in 6 seconds. From plugin UI, it runs in 5 minutes.

Stall time does vary with complexity. A 20mm square board with just one SMD transistor panelized 3x4 takes less than 2 secs for CLI, but 7 secs from plugin UI, so hard to notice much difference.

I have seen this symptom on all of the four boards I tried, on two different PCs, with significantly different kikit parameters (eg: mousebite tabs vs vcuts). I did not see any boards on which kikit UI ran without stalling.

When launched from GUI, the progress bar advances to a certain point (position varies: one example boards stalls at say 10%, another say 40%), then stops there for the majority of the several minutes, then suddenly sprints to 100%.

Inspecting the stalled kikit in Windows Resource monitor, there's no high disk activity (ie: no obvious virtual memory disk thrashing). CPU is about 12%, which is 100% of one "logical processor" on my "4-core, 8 logical processor" i7-6700 machine.

Steps to Reproduce

Select a candidate panelization-friendly PCB, say 60x60mm, with 10 ICs and several dozen other components. I don't know if it matters, but all my sample boards had at least one QFP-44.
Configure a kikit config json file to create a panel of say 3x3 boards.
Run this from the kikit plugin GUI > Panelize button.
Observe stall of several minutes
Compare to speed running from CLI, likely just a few seconds.

Of course, if you don't see the stall, then there's some factor outside this repro setup that's somehow in play.

yaqwsx commented 1 month ago

Could you try the upstream version? There were some significant changes in the way GUI operates, so I want to be sure the problem is present.

The only difference between CLI and GUI is that after successful penalization, GUI has to copy the panel into the currently open window. It is possible there might be some inefficiency there, I will check it out.

gwideman commented 1 month ago

Could you try the upstream version?

I can try that, but it will probably take me a day to get to it.

gwideman commented 1 month ago

OK, I have not tried anything new yet.

I see that there's a new version of Kicad just released (8.0.4). I should probably install that to make any further tests relevant.
I see you uploaded revised KiKit to PyPi. That's the backend, right? And shared between CLI and GUI. I can easily install it of course, but I think we're not suspecting that as the culprit, right?
I believe it's the GUI plugin that is most suspicious at the moment. I did not see an updated plugin in Plugin and Content Manager yet. Is that imminent?
If you still would like me to "try the upstream version", I realize I don't know how to do that. I'm happy to try it if you could point me to a procedure to install the upstream version of whatever is the thing you'd like me to test.

yaqwsx commented 1 month ago

The GUI is part of the backend. The KiCAD plugin is just a tiny loader. Definitely try to reproduce this with the v1.6.0. At the moment, there is no point in trying upstream version as it is the same as the just released v1.6.0.

shuki25 commented 1 month ago

I have an exact same problem. It stalled after hitting on the panelize button. I didn't have that issue until I upgraded KiCad to 8.0.4 and kikit to 1.6.0. I even upgraded your GUI backend to 1.6.0. The overall CPU when stalled is at 3-4%.

shuki25 commented 1 month ago

I want to add that it stalls on BOTH Windows version and Mac OS (silicon) Ventura 13.5.1. When hitting panelize button, it crashed. Looks like a segmentation fault as it was trying to access to memory outside of memory region for the process.

Although the panelization was partially done. When I reopened KiCad, all I can see was a generated Edge.Cuts layer but not the rest of the design with all other layers. See screenshot:

Although I could panelize it without any issues if I run from CLI. Both were on KiCad 8.0.4 and I've updated to the most recent versions of kikit (shell command) and kikit-gui (KiCad plug-in).

Below is the crash report:

Process:               pcbnew [33202]
Path:                  /Applications/KiCad/KiCad.app/Contents/Applications/pcbnew.app/Contents/MacOS/pcbnew
Identifier:            org.kicad.pcbnew
Version:               8.0.4 (8.0.4)
Code Type:             ARM-64 (Native)
Parent Process:        kicad [31371]
Responsible:           kicad [31371]
User ID:               501

Date/Time:             2024-07-20 12:20:18.5048 -0400
OS Version:            macOS 13.5.1 (22G90)
Report Version:        12
Anonymous UUID:        94DCB19B-8018-E78F-AD75-5D423B7ED59F

Time Awake Since Boot: 840000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x2626269602020207 -> 0x0000269602020207 (possible pointer authentication failure)
Exception Codes:       0x0000000000000001, 0x2626269602020207

Termination Reason:    Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process:   exc handler [33202]

VM Region Info: 0x269602020207 is not in any region.  Bytes after previous region: 41944684298760  Bytes before following region: 63127395630585
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      commpage (reserved)        1000000000-7000000000   [384.0G] ---/--- SM=NUL  ...(unallocated)
--->  GAP OF 0x5f9000000000 BYTES
      MALLOC_NANO              600000000000-600008000000 [128.0M] rw-/rwx SM=PRV  

Kernel Triage:
VM - (arg = 0x0) pmap_enter retried due to resource shortage
VM - (arg = 0x0) pmap_enter retried due to resource shortage
VM - (arg = 0x0) pmap_enter retried due to resource shortage
VM - (arg = 0x0) pmap_enter retried due to resource shortage

gwideman commented 1 month ago

Update to this issue, using newly updated:

Kicad 8.0.4
kikit 1.6.0 plugin installed from Kicad Plugin and Content Manager
kikit 1.6.0 installed via pip and PyPi

In brief testing I was not able to get kikit to complete error free. Here's a list of what I noted along the way.

In kikit's "Panelize a board" dialog, the Input File edit box slot is missing. So the GUI no longer shows what file you have selected (though you can read the CLI command to figure it out.)
There's a new Output File button. It also is missing an edit box showing the current selection. But more to the point -- does this represent a change of workflow, or that kikit will save the output file? Because at the moment, having kikit NOT save the output file is good because it supports iterating on the parameters and generating the panel into the open Kicad PCB Editor window to check the results. Maybe the idea of the Output File was just to show the composed CLI command?
With a source PCB and JSON file (that I had used in a previous run, hence comparable), I tried running the Panelize button.
- The "Running kikit" dialog appears, progress bar advances to perhaps 10% then stops.
- After 10 secs, the kikit UI flashes and the titlebar changes to "Running kikit (Not Responding)".
- After 2:40, an error dialog appears "Cannot perform: 'PCB_TEXT' object has no attribute 'GetWidth' " The previously-stalled progress bar changes to rapidly scanning bar ( "indeterminate mode").
- If I dismiss that error dialog, the progress bar abruptly goes to 100%, and a panel appears in PCB Editor. But I don't know if it's actually complete, given the error condition that likely interrupted composition of the panel.
- If I don't dismiss that error dialog, but instead click on either the kikit or PCB Editor window, the error dialog disappears behind and now there's no way to proceed. Neither kikit not PCB Editor respond to UI interaction, and there's no way I could find to bring the error dialog to the front to dismiss it. So this requires killing Kicad from Task Manager.

Conclusions:

The lengthy hang that's the main topic of this issue does not appear to be solved by the Kicad or kikit updates.
I have mentioned some other issues in Kikit that may not be directly pertinent to the current issue, and if warranted I can create separate issues..

JSON settings used:

{
    "layout": {
        "hspace": "0.mm",
        "rows": "3",
        "cols": "4"
    },
    "tabs": {
        "type": "full"
    },
    "cuts": {
        "type": "vcuts",
        "clearance": "0.2mm",
        "layer": "Edge.Cuts"
    },
    "framing": {
        "type": "frame",
        "hspace": "0mm",
        "vspace": "0mm",
        "width": "10mm",
        "maxtotalheight": "300mm",
        "maxtotalwidth": "300mm"
    },
    "text": {
        "type": "simple",
        "voffset": "3mm",
        "text": "[REDACTED]        07 JUL 2024       2024 [REDACTED]       STEP= xxxx"
    },
    "post": {
        "origin": "bl",
        "dimensions": "True"
    }
}

elessargr commented 1 month ago

+1 also impacted from this. Dont know why its getting stuck/stalls which in the end crushes the KiCad.

Can we download the 1.5.x version (unofficial upstream version) that was before the 1.6.0 release? For me that was working fine.

gwideman commented 3 weeks ago

Conversation on this issue seems to have stalled. Anything we can do to help diagnose this issue?

yaqwsx / KiKit