Crash when saving layout

[mattmunee]

The plugin is crashing when trying to save a layout. Based on the attached log, it appears that the plugin is expecting the schematic to be saved in the /tmp directory along with the layout, but only the layout is saved into the /tmp directory. I've tried this on 3 different projects, and all of them behave the same. Log is attached.

09-06 15:26:41 com_github_MitjaNemec_SaveRestoreLayout.action_save_restore_layout 172:Plugin executed on: 'linux'
09-06 15:26:41 com_github_MitjaNemec_SaveRestoreLayout.action_save_restore_layout 173:Plugin executed with python version: '3.11.4 (main, Jun  7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)]'
09-06 15:26:41 com_github_MitjaNemec_SaveRestoreLayout.action_save_restore_layout 174:KiCad build version: 7.0.7-2.fc38
09-06 15:26:41 com_github_MitjaNemec_SaveRestoreLayout.action_save_restore_layout 175:Plugin version: 2.0.8

09-06 15:26:41 com_github_MitjaNemec_SaveRestoreLayout.action_save_restore_layout 176:Frame repr: <wx._core.Frame object at 0x7f88982c0f70>
09-06 15:26:41 com_github_MitjaNemec_SaveRestoreLayout.action_save_restore_layout 194:Anchor footprint reference is 'A66'
09-06 15:26:42 com_github_MitjaNemec_SaveRestoreLayout.action_save_restore_layout 203:Save layout chosen
09-06 15:26:42 com_github_MitjaNemec_SaveRestoreLayout.save_restore_layout 354:Working on /home/mreaves/kicad-projects/big-board/big-board.kicad_pcb
09-06 15:26:42 com_github_MitjaNemec_SaveRestoreLayout.save_restore_layout 132:getting a list of all footprints on board
09-06 15:26:42 com_github_MitjaNemec_SaveRestoreLayout.save_restore_layout 359:Saving the current board temporary in order to leave current layout intact
09-06 15:26:42 com_github_MitjaNemec_SaveRestoreLayout.save_restore_layout 365:Saving board as tempfile: /tmp/temp_board_file_for_save.kicad_pcb
09-06 15:26:43 com_github_MitjaNemec_SaveRestoreLayout.save_restore_layout 370:Loaded temp boardfile: /tmp/temp_board_file_for_save.kicad_pcb
09-06 15:26:43 com_github_MitjaNemec_SaveRestoreLayout.save_restore_layout 371:Get project schematics and layout data
09-06 15:26:43 com_github_MitjaNemec_SaveRestoreLayout.save_restore_layout 132:getting a list of all footprints on board
09-06 15:26:43 com_github_MitjaNemec_SaveRestoreLayout.action_save_restore_layout 208:Fatal error when creating an instance of SaveLayout
Traceback (most recent call last):
  File "/home/mreaves/.local/share/kicad/7.0/3rdparty/plugins/com_github_MitjaNemec_SaveRestoreLayout/action_save_restore_layout.py", line 206, in Run
    save_layout = SaveLayout(board, src_anchor_fp_ref)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mreaves/.local/share/kicad/7.0/3rdparty/plugins/com_github_MitjaNemec_SaveRestoreLayout/save_restore_layout.py", line 372, in __init__
    self.save_prjdata = PrjData(self.board)
                        ^^^^^^^^^^^^^^^^^^^
  File "/home/mreaves/.local/share/kicad/7.0/3rdparty/plugins/com_github_MitjaNemec_SaveRestoreLayout/save_restore_layout.py", line 175, in __init__
    self.parse_schematic_files(self.sch_filename, schematic_found)
  File "/home/mreaves/.local/share/kicad/7.0/3rdparty/plugins/com_github_MitjaNemec_SaveRestoreLayout/save_restore_layout.py", line 190, in parse_schematic_files
    with open(filename, encoding='utf-8') as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/temp_board_file_for_save.kicad_sch'

[mattmunee]

I've traced this a bit further. In "save_restore_layout.py" line 170:

 if (len(unique_sheet_ids) > len(self.dict_of_sheets)) and not dont_parse_schematics:

For some reason, len(unique_sheet_ids) equals 38 and len(self.dict_of_sheets)) equals 37. I've found another project where the plugin works as expected and the lengths are equal. I have not been able to track down why these values are not equal in the projects that cause the crash.

For now, I'm just bypassing this statement by ensuring it's False:

  if (len(unique_sheet_ids) > len(self.dict_of_sheets)) and not dont_parse_schematics and False:

[MitjaNemec]

Yeah, I'll have to test this logic a bit further. The main issue is that for most of the designs the plugin can figure out the schematics hierarchy from footprint properties. But for project where you have nested sheets and one sheet only contains other sheets and no symbols the the plugin has no footprints on this level and it fails to get the schematics hierarchy. In this case it will start to parse the schematics. It looks like I have a bug when the plugin saved such a project but it fails on restore as it tries to parse the schematics as it is obviously nonexistent.

Also be advised, my schedule for September is quite full so you'll probably need to wait for bugfixes. For me, it's not that urgent. Thank you for watching this

[mattmunee]

Understood. If I have time, I'll try to propose a proper fix. This plugin is an absolute lifesaver for my current work.

[MitjaNemec]

This should be fixed in 2.1.1. Can you please install it from file and test it before I push it to be available via PCM.

[mattmunee]

This should be fixed in 2.1.1. Can you please install it from file and test it before I push it to be available via PCM.

Well the "save" function worked as expected this time. The "restore" function still pops up an error that says "list index out of range". However, it appears to restore the layout anyway. The log is attached.

(Hopefully I'm not convoluting things. My design has changed a lot since this originally happened, and the existing plugin 2.0.8 didn't crash when I tried. I reverted my design back to the day before I started this issue thread and tried from there. Plugin 2.0.8 still crashed as it did back in September.) save_restore_layout.log

[MitjaNemec]

Hmmm, this looks like a different bug. the plugin probably failed to restore some zones. Can you hold on to this design? If you can not share it, I'd like to add a bit of logging to help me narrow it down and hopefully fix it.

[mattmunee]

Unfortunately I cannot share the design, but I'm happy to help however I can.

[MitjaNemec]

No worries, we'll do the debugging remotely. It might be a bit tedious but we should be able to make it through.

So can you rerun the save and restore part with 2.1.2. And for the restore part please attach the log file.

[mattmunee]

Here's the new log with the error. However, I've played around and realized that this depends on where the components are on the layout when the layout pckl is saved. When I generated the error, the components were in the default position off the board when I ran "Update PCB from Schematic". That button will just drop all of the components way off to the right of the board (which is pretty large, like 10x10 inches). If I move all of the components to the center of the board outline, then highlight and "save" the layout pckl, when I restore I do not receive an error. Still kind of annoying that I can't save/restore wherever I want, but there is a workaround. save_restore_layout.log

[MitjaNemec]

Thanks for the log. So from looking at the log it seems that the saved design includes a zone that was originally connected to the net GRP_PWR2.VDDA. But this net does not exists in the destination (target) project.

So I have a few questions:

1. Can you confirm that this net exist in the source design?
2. Can you confirm that there is a zone connected to this net in the source design?
3. If you try to save the design, is this zone highlighted by the plugin, to let you know that it will be saved?
4. Is this net connected (not electrically but in the general sense) anyhow with the hierarchical sheet that you want to save the design for?
5. How would you want the plugin to behave in this case? Should the zone be saved? If it should, how should the plugin handle the restore part?

[mattmunee]

I apologize for the delay. This actually has me pretty confused. I don't see why GRP_PWR2.VDDA should be in this particular hierarchical sheet. Would you mind if we move this conversation to email? I wouldn't mind sharing some screenshots with you, but I'd rather not post them to the entire world. I'd also be open to a Zoom conversation or otherwise so we could interactively try some things.

[mattmunee]

I can see differences in the highlighted items when doing "save", depending on whether all items are within the board outline or not. In one case, if I left four resistors outside the left edge of the board (all other components of the hierarchical sheet are within the board outline), then "save" highlights some zones that seem incorrect, and the restore fails with the "list index out of range" error. If I move those 4 resistors inside the board outline, so that now all components from the hierarchical sheet are within the board edge, then "save" does not highlight those zones, and I get no error when restoring the layout.

[mattmunee]

Attached are the restore logs for the good and the bad case. save_restore_layout_bad.log save_restore_layout_good.log

[mattmunee]

Here are the "save" logs, which look identical to me. save_restore_layout_SAVE_bad.log save_restore_layout_SAVE_good.log

[MitjaNemec]

Sure, you can reach me at mitja.nemec@fe.uni-lj.si. As for a zoom meeting currently I think that with my schedule it might take a while before we find a slot where we both are available. Currently I am in UTC+2 time zone. I don't think that we need to meet right now as I've got some info which should help me narrow down the issue.

then "save" highlights some zones that seem incorrect, and the restore fails with the "list index out of range" error.

This leads me to believe that the issue is in the save part of the plugin, and the plugin saves also the object which are within the bounding box defined by the footprint from the saved sheet, but the objects are not connected to any net from this sheet.

I've just look at the code and there is no logic filtering the zones. So I'll need to add the logic which will filter the zones so that from the zones which are on copper layers and are connected to some net only zones which are connected to the net that exists within the source sheet are selected.

Now this is way easier to write than implement so it might take some time.

Thanks for the help with the diagnosis.

[MitjaNemec]

P.S. Can you do an experiment for me.

Can you place the resistors so that zones which should not be selected are selected. Then you remove the zones, but you place a couple of tracks there. Then you do the save and restore part. and report back the results.

I have a hunch that there is a same bug present for tracks as well as for zones

[mattmunee]

I believe your hunch is correct. Attached are the save and restore logs. I noticed that other tracks were highlighted as well, and while playing around with it, it does appear to be dependent on the bounding box of the components within the selected hierarchical sheet.

save_restore_layout_RESTORE.log save_restore_layout_SAVE.log

[MitjaNemec]

But were the tracks that were selected (but should not be) also replicated?

[mattmunee]

No, they were not actually replicated. They were highlighted during "save", but not seen when doing "restore".