layout2fc: a python script to import a 3D structure from klayout (work in progress)

[wsteffe]

layout2fc.py.zip. Usage: layout2fc.py -stack stack_file dxf_file

Following an example of stack file (PH10 is a UMS process for GaAS MMIC). Format is: layerNum/LayerDataType : zstart zstop #user layer name (not parsed) File extension must be .stack.

"UMS_PH10.stack":
scale:  0.001 0
10/0:  0.0 70.0              # TR
4/0:  70.0 71.0              # N1
6/0:  70.0 71.0              # RM
16/0: 70.0 71.0              # RHWH
5/0:  71.0 71.19             # DPC
40/0: 71.19 74.19            # EL
40/2: 74.19 77.19            # EL2
40/1: 77.19 80.19            # EL1

First line is a scale factor. In my project the stack heights are in micron and I have assumed that FC unit is mm. Same scaling must be set also in the FC dxf Import Options.

I am sorry I may not share the dxf file of my project but it is quite easy to make a test case with klayout.

Works is still in progress. I have made the extrudes but I not able to set visibility of pads. The lines 91 and 112 of this script have no effect:

91: body.Visibility = True
112: pad.Visibility =True

PS The script has been improved with respect with first snipped posted at https://github.com/realthunder/FreeCAD/issues/419. Now it uses the python module klayout.db instead of pya which is deprecated because this name was already taken by another project. On ubuntu the klayout module is made available with the installation of "klayout_0.27.9-1_amd64.deb".

Any help to improve this script would be appreciated. Thanks.

[wsteffe]

Hi all, having looked around, it seems to me that it is not possible to make visible objects with a console application because ViewObject is not available. So I would like to know if, at least, it would be possible to have a macro which, after opening the FC document, can set obj.ViewObject.Visibility=True for all obj with obj.Visibility=True.

Another problem I have with the current script (here I am uploading the last version) is that planes and axes (udsed by sketches) are lost in the model construction and I get a lot of error messages like:

0.000354148 Selection.cpp(1964): Sub-object Stage_1_2_3_LNA_layout_EM#DPC.Y_Axis. not found
0.00052545 Selection.cpp(1964): Sub-object Stage_1_2_3_LNA_layout_EM#DPC.Z_Axis. not found
0.000678069 Selection.cpp(1964): Sub-object Stage_1_2_3_LNA_layout_EM#DPC.XY_Plane. not found
0.000830951 Selection.cpp(1964): Sub-object Stage_1_2_3_LNA_layout_EM#DPC.XZ_Plane. not found
0.00097343 Selection.cpp(1964): Sub-object Stage_1_2_3_LNA_layout_EM#DPC.YZ_Plane. not found
0.0124574 Selection.cpp(1964): Sub-object Stage_1_2_3_LNA_layout_EM#TR.X_Axis001. not found
0.0126192 Selection.cpp(1964): Sub-object Stage_1_2_3_LNA_layout_EM#TR.Y_Axis001. not found

I would like to put in the bodies also planes and axes used by sketches (which now are deleted).

[wsteffe]

I forgot to upload the last version.Here is is: layout2fc.py.zip Very small difference with respect to previous one:

• Body placement is used instead of pad offset.
• A recompute() is added at the script end.

[wsteffe]

test_case.zip Hi RT, may you please try to convert this test layout. Then please open it in FC and make a "select all". You will see many of such errors in the command line:

0.000411892 Selection.cpp(1964): Sub-object test_case#DPC.Y_Axis. not found
0.000593112 Selection.cpp(1964): Sub-object test_case#DPC.Z_Axis. not found
0.000747768 Selection.cpp(1964): Sub-object test_case#DPC.XY_Plane. not found
0.00089718 Selection.cpp(1964): Sub-object test_case#DPC.XZ_Plane. not found
0.00104942 Selection.cpp(1964): Sub-object test_case#DPC.YZ_Plane. not found

I am starting to think that it is a FC bug because these objects are visible in the tree view. Why FC can't find them ?

[wsteffe]

Now I am sure that it is a FC BUG. In fact the same kind of error is generated with any other document created by me directly in FC (without using this script). I never observed it because I had never used the "Select All" command.

[wsteffe]

Hi RealThunder, I would like to export into a step file from my script. Here annexed the last version with a first trial to export a part with all related bodies. layout2fc.py.zip

The problem is that only the tip is exported (that is just one pad per body). In the GUI it is possible to force the export of other body objects with Toggle Export command. But how is it possible to get the same result in a python script without the GUI ? I tried to make a list of objects to export. You may see these code lines in the script commented out because not working. In fact the document structure is not preserved.

#expObjs =[part]
#for ob in doc.Objects:
#   if ob.TypeId=="PartDesign::Body" or ob.TypeId=="PartDesign::Pad":
#      expObjs.append(ob)

[realthunder]

test_case.zip Hi RT, may you please try to convert this test layout. Then please open it in FC and make a "select all". You will see many of such errors in the command line:

0.000411892 Selection.cpp(1964): Sub-object test_case#DPC.Y_Axis. not found
0.000593112 Selection.cpp(1964): Sub-object test_case#DPC.Z_Axis. not found
0.000747768 Selection.cpp(1964): Sub-object test_case#DPC.XY_Plane. not found
0.00089718 Selection.cpp(1964): Sub-object test_case#DPC.XZ_Plane. not found
0.00104942 Selection.cpp(1964): Sub-object test_case#DPC.YZ_Plane. not found

I am starting to think that it is a FC bug because these objects are visible in the tree view. Why FC can't find them ?

This is indeed a bug, and is fixed in 8262c84c71ae74a2f2b06f4e59197c7225eb7e27. It's not really a critical bug though.

[realthunder]

The problem is that only the tip is exported (that is just one pad per body). In the GUI it is possible to force the export of other body objects with Toggle Export command. But how is it possible to get the same result in a python script without the GUI ? I tried to make a list of objects to export. You may see these code lines in the script commented out because not working. In fact the document structure is not preserved.

To enable selective export in a group container, you first need to set, assuming we are talking about Body as the group here, Body.ExportMode = 'Child Query'. And then add a dynamic property to the object you want to export, e.g.

Pad.addProperty('App::PropertyBool', 'Group_EnableExport', 'Group')
Pad.Group_EnableExport = True

[realthunder]

BTW, unless you have a good reason to keep the hierarchy structure, or want multiple face color, it would be easier to just export the shape as step.

Pad.Shape.exportStep(filename)
# or to group multiple shapes
Part.makeCompound([Pad, Pocket]).exportStep(filename)

[wsteffe]

Yes the bug was not critical. Bu I was experimenting with scripting (which I do not know well) and seeing so many error messages had let me think that I was doing something wrong.

[wsteffe]

Adding

Body.ExportMode = 'Child Query'
....
Pad.Shape.exportStep(filename)
Part.makeCompound([Pad, Pocket]).exportStep(filename)

was a good thing. In fact Pads in the Bodies were toggled for exports. But they were not really exported in the script because all are hidden. I have to open the document in FC GUI, make "select all" and "unhide" before exporting again. It is a pity that it is not possible to make visible objects without GUI. I could change my FC import-export/STEP preference to export also invisible objects but I do not like very much this solution.

[wsteffe]

unless you have a good reason to keep the hierarchy structure At least I need separate bodies. Each body includes the content of a PCB layer which has a material property. In example a layer may be a dielectric and another one may consist of metal vias protruding into the dielectric. In this case the stack z (zstart and zstop) associated with the DIEL_layer and with the VIA_layer may be the same. I would like to use the data type (another integer value associated with the layer) to define a sort of insertion priority. VIA_layer data type should be higher than DIEL_layer data type meaning that the first one is inserted into the second one. In other words the VIA_layer sketch has to be used two times: a positive pad and a negative one (a hole) in the DIEL_layer. This is the logic I would like to implement. To avoid unnecessary operations negative operations should be performed only when the two stack intervals overlap.

[wsteffe]

I forgot to say that I am assuming that the layout structure is defined in such a way that layers with same data type do never overlap.

[wsteffe]

Hi RT, I have made a couple of experiments (see annexed version 2 and 3 of the script) layout2fc_v2.py.zip layout2fc_v3.py.zip v2 is bases on the first option suggested by you:

Body.ExportMode = 'Child Query'
....
Pad.Shape.exportStep(filename)
Part.makeCompound([Pad, Pocket]).exportStep(filename)

As told before the structure is preserved but I have to manually make visible all pads in GUI before exporting the step file.

With v3 I have tried to make a separate compound of all pads in each body. The generated structure seems ok when I am looking at the generated file "test_case.FCStd":

But the names are mangled and the structure not preserved in the exported step file:

It seems to me that step export doesn't deal well with bodies containing compounds. Or may be I am doing something wrong. I am not very sure about the following lines:

   comp=doc.addObject("PartDesign::Feature","Compound")
   shape=Part.makeCompound(pads)
   comp.Shape=shape

[wsteffe]

Hi RT, I am starting to think that step export is broken in some way. I have just done a new version of my script (here annexed) which merges all the layer sketches into a single one. layout2fc_v4.py.zip Now there is a single pad in each body but I imagine that, internally, FreeCAD must associate a compound to that pad. The generated FC document (which koks nice to me) is the following: test_case.FCStd.zip

But if you export the Part into a step file you will see that the body names and their content are not preserved. The result is the same if you use the step file generated by the script or a step file exported in the GUI. This happens with the internal exporter while if I set the legacy exporter FC crashes.

[wsteffe]

To better clarify I would like to say that my problem is related with the generation of names for the new entities in the step file. From what I have seen I may understand that the exporter needs to split the compound and find new names for its elements. That is ok for me but I would like that the name assigned to a body (by the user) is kept for the body and not reassigned (by the step exporter) to the first element of the body. My application requires a stable body name because this name is associated with a material/electrical property. If the name changes the association is broken and the electromagnetic simulation fails.

[wsteffe]

Hi, I have just discovered that the name mangling was not done in the step export but int its re-importation (for visualization). This problem disappears enabling STEP compound merge in the Step Import Preferences.

Following are the screenshot associated with the documents "tests_case.FCStd" and "tests_case.step":

These documents were generated by the last version of the script layout2fc.zip (here updated together with input files) using the command: layout2fc -stack UMS_PH10.stack test_case.dxf

In this last version I have implemented the insertion logic based on data type number explained above and I have updated the test case to apply this logic: Layer wth name DIEL_GaAs has a data type=0. All other are metals layers and have data type=1. So they are inserted in layer DIEL_GaAs when they overlap with it. The only one which actually overlaps is BC_TR which describes a via hole going through DIEL_GaAs up to the ground (bottom face of MMIC). Some layers are not written into the step file because are void in this exercise.

[realthunder]

So, no problem now?

Anyway, just a few notes about the OCC compound and STEP export/import. Any hierarchy below the compound will not be exported, because OCC treat compound as simple flattened shape, and won't go under for its children. The importer has the option to explode the compound with at most one level of its children, with some generic names, because the export didn't actually export those children individually.

[wsteffe]

Yes, it is ok now and thanks for the note which actually explains the behaviour I am seeing. But it lets me think that the import option "Enable STEP compound merge" is wrongly named. It should be named "Disable STEP compound explode". In fact it is when I uncheck this option that I see the childrens of my multi-pads in the re-imported step file and that body names are changed. There is another import option named "Expand compound shape" but it seems to me that it doesn't have any effect (at least with my test case).

[wsteffe]

Hi RT, any idea about why in the FC document generated by the annexed script, the reference axes/planes of part "CMP_test_case" are named "unnamed" ?

layout2fc.zip test_case.zip .

[realthunder]

The problem is fixed in 6f122c87d050e0cd917b25ae029816fd6c2d87e9

BTW, can you please put your file into a github repo, so that we can track its changes? And why do your script's indentation is kind of mixed up? Anywhy, this is my modification to your script with some clean up. Also allows it to be run either as FreeCAD Macro or a standalone script.

layout2fc.zip

[wsteffe]

Hello RT, many thanks for your clean up and improvements. I have put it at https://github.com/wsteffe/layout2fc. A couple of lines are changed from your last revision.

[wsteffe]

Hi RT, I have extended the stack format adding two columns holding (for each layer) the kind of operation and the insertion order. In the previous version I was using layer datatype to define the insertion order but I think that it is better to have a separate data field. The new format is better explained in following lines taken from the file UMS_PH10.stack stored in the github repository.

scale:  0.001                    
49/0:  0.0 70.0  add  0   # DIEL_GaAs
10/1:  0.0 70.0  ins  1   # BC_TR
4/1:  70.0 71.0  ins  1   # BC_N1
61/0:  0.00 100.00 vsurf 1  # SPLIT_1

The first line with scale is used to convert FreeCAD unit (mm) into micron. The line: 49/0: 0.0 70.0 add 0 # DIEL_GaAs means that layer 49/0 defines a vertical solid extrusion from h=0 to h=70mu, the last number (before the comment) refers to the ordering of the related CAD operation. The order is important for the ins (insert) operation which consists of a solid extrusion plus a cut of all previously created solids which intersect the new one. The last line should produce a vertical surface (instead of a solid object) using the sketch profile stored in layer 61/0 of the updated test_case.dxf. This operation is implemented in the function addVSurf(sketch,h) defined starting at line 86 of the script and based on the extrude of Part WB. But there is a problem because the obj of type Part::PartFeature returned by this function can not be assigned to the related body at the line 174 of the script:

174: body.addObject(obj)

Not yet tested but the same kind of problem will probably affect also the function addHSurf(sketch) defined starting at line 96 of the script and based on makeFace of Part WB. Vertical surfaces (generated by 'vsurf' operation) may be used to define waveguide ports and of split surfaces. The latter are used by my EmCAD solver to define a subdomain decomposition. Horizontal surfaces (generated by 'hsurf' operation) may be used to approximate thin metal stripes. This approximation allows to achieve a better aspect ratio of tetrahedral meshes.

[wsteffe]

I have fixed the problem related with body.addObject(obj) for objects created by function addVSurf(sketch,h).

But I encountered another problem related with the kind of data (a single line) included in layer 61 of the updated test_case.dxf. This kind of data is properly recognized when I am using import dxf from FC GUI. But the sketch created at line 154 of the script:

sketchi = Draft.make_sketch(skObjs, autoconstraints=True)

doesn't include any edge or wire.

The dxf import at line 146 creates a new object of type Part::PartFeature (instead of Part::Feature)

Any idea how to address this problem ?

[wsteffe]

Now (see at https://github.com/wsteffe/layout2fc) I have fixed the sketch creation with object of type Part::PartFeature and also the fillowing function which uses it:

    def addVSurf(sketch,h):
        extrude=doc.addObject('Part::Extrusion','Extrude')
        extrude.Base = sketch
        extrude.DirMode = "Normal"
        extrude.DirLink = None
        extrude.LengthFwd = h
        extrude.LengthRev = 0.000000000000000
        extrude.Solid = False
        extrude.Reversed = False
        extrude.Symmetric = False
        extrude.TaperAngle = 0.000000000000000
        extrude.TaperAngleRev = 0.000000000000000
        extrude.Visibility =True
        return extrude

But still there is a problem because I do not know how to wrap the Part::Extrusion into a PartDesign::Body. If I do that from the GUI the Part:Extrude is automatically inserted into the active body. But the script produces what shown in the following image

I would like that the last Item named "Shell_Sketch_Split_1" were placed in the Body named SPLIT_1 and, possibly, with export option toggled on. RT, may you please help me on that point ?

[realthunder]

You should add the sketch to the body first. And there is an Extrude feature in PartDesign, PartDesign::Extrusion. It is in fact derived from Pad. The only difference is that it is easier to create non-solid.

[wsteffe]

Thanks RT, it worked as you said (fixed included in last commit). But there are two minor issues. 1) The export setting done in the following lines

          obj.addProperty('App::PropertyBool', 'Group_EnableExport', 'Group')
          obj.Group_EnableExport = True

was not visible after opening the generated FC document. But I think it was actually set because I had to toggle it two times (not one) to make it visible as in the image below.

2) Minor. In the structure Tree the sketch is not inside the Extrude as it is for Pad.

Last question: is there also a PartDesign::makeFace which can be used to fill a sketch into an "horizontal surface" placed in a PartDesign Body ?

[wsteffe]

Last commit to fix the setting of export of non solids (added body.ExportMode = 'Child Query').

[wsteffe]

Anyway the vertical surface in body SPLIT_1 is still not exported into the step file at line 201 of script:

Import.export([part], fname1+".step")

On the other side, it is exported from the generated FCStd after unhiding all objects.

[realthunder]

Your code looks fine. Do you have a layout file that I can test?

As for making face, just create a PartDesign::SubShapeBinder, and set the sketch to its Support. It will make a face.

[wsteffe]

Ah thanks, I didn't read in time your post and I have just made a commit using Part::Face in function addHSurf (which is not well integrated with PartDesign Body). I will try SubShapeBinder as you said. layout and stack are test_case.dxf and UMS_PH10.stack in test folder of my repo. I would like to have a better looking layout but I had no time to do it.

[wsteffe]

Hi RT, but SubShapeBinder Support doesn't accept objects of type Sketcher::SketchObject I have tried with:

def addHSurf(sketch):
        obj=doc.addObject('PartDesign::SubShapeBinder','Binder')
        obj.Support =sketch
        obj.Visibility =True
        return obj

but I get the following error:

  File "/home/walter/local/EmCAD/bin/layout2fc", line 103, in addHSurf
    obj=doc.addObject('PartDesign::SubShapeBinder','Binder')
TypeError: {'sclassname': 'N4Base9TypeErrorE', 'sErrMsg': 'Invalid type. Accepts (DocumentObject, (subname...)) or sequence of such type.', 'sfile': '', 'iline': 0, 'sfunction': '', 'swhat': 'Invalid type. Accepts (DocumentObject, (subname...)) or sequence of such type.', 'btranslatable': False, 'breported': False}

Line 103 is that one with "obj.Support =sketch"

[wsteffe]

I have seen that obj.Support is a list. So I have tried with: obj.Support.append(sketch) or with: obj.Support.append(sketch.Label)

This time I do not get any error but Support is not set in the generated FCStd document. I may set it in the GUI and in that way I may get the face I wanted to have.

Macro recording for this operation gives:

FreeCAD.getDocument('test_case').getObject('Binder').Support=FreeCAD.getDocument('test_case').getObject('Sketch008')

So it seems that it should be OK to assign (not to append) the sketch object to Support. But my python script can't do that.

[realthunder]

Most of the time, you can't directly call property's method. The value returned by the property is detached, so changing the returned value by calling its method won't affect the property. Simply assign Support with the object. I just tried in FC Python console and it works fine. No idea why it didn't work on your side.

One thing though, the line 103 in the error message does not seem to be the one causing the error. Just my guess here, did you modified the code, but forgot to reload the module?

import importlib
importlib.reload(layout2fc)

[wsteffe]

I was running the script alone in a bash console and not in the FC console.

[wsteffe]

I have no idea how I can run a python script with arguments from FreeCAD GUI. Macro/Macros/Execute may execute a python script but without any argument. And RT, what have you done to create a "layout2fc" module that can be imported in FC console with importlib.reload(layout2fc) ? And once you have imported it, how do you use it with arguments ?

[realthunder]

Your script already has my modification to accept argument in a function called main(tech_fname, fname). Just copy layout2f.py to your macro directory. For me, it is in ~/.FreeCAD/Macro. If you are on Linux, then it is easier to just create symlink to the file, instead of actual copy. After that, you can run the script in FreeCAD Python console

import layout2fc
layout2fc.main(tech_fname, fname)

But you need to remember, once you've made any changes to your script, without quitting FreeCAD, you need to reload the script to see your changes

import importlib
importlib.reload(layout2fc)

[wsteffe]

Just made last commit (addHSurf implemented with subShapeBinder and small change in calling syntax from bash). Now I have tried using it from FC console. But I get same error as calling the script from bash console. See image:

[wsteffe]

addHSurf problem was fixed in the last commit by adding "doc.recompute()" just after body.addObject(sketchi) and before obj=addHSurf(sketchi)

[wsteffe]

Now that the script is working as expected I would like to integrate it with FC GUI. So I have added two new files (layout2fcGui.py and layout2fc.ui) for that. But I have little experience with Qt creator and I do not know how to put a file browser in task panel. Pressing browse button below DXF File should trigger a file browser based on QFileDialog but it doesn't work.

Any idea ?

[realthunder]

I have submitted a PR for the fix to your repo

[wsteffe]

Hi RT, I have just tried to use the GUI IF and it works very nice after your commit. Many thanks. I have added also an Icon and I have customized my PartDesign menu with a new command named "Layout Import" that uses this icon. What do you think about including all this stuff in your Repo/Branch to make it available to all users ?

[wsteffe]

It works but it is very inefficient. Not usable with a complex layout. I have just added a new (more realistic) test case given by files test_case2.dxf and test_case2.stack. It takes hours to import it. The problem seems to be _Draft.makesketch which takes a very long time to recognize closed wires. Klayout internal code does it in less than a second. And in the same time it can also do the merging of overlapping contours (a sort of 2D boolean) .

[realthunder]

What do you think about including all this stuff in your Repo/Branch to make it available to all users ?

Yes, sure. But let's make it more functional first. I'll check your new test case soon.

[realthunder]

Some questions about your script. Do you intend to modify the imported layout using FreeCAD? If not, building shapes using body is just wasting time. It will be much faster to just build the shape without creating object.

[realthunder]

Another question, is there any native file format for klayout? It seems that when exporting dxf, klayout discretize circles into tiny edges, which causes massive slow down in wire and face making.

[wsteffe]

I do not need to modify it but I like to have each layer shape in a separate body because of data structure created in the step file. Each body becomes a named product taking its name from the layer. The name plays an important role in EmCAD. In example a shape with name starting with DIEL_ is recognized as a dielectric and you may assign a dielectric property to it. Furthermore this association is done using the name as a key. You may change the step file and reload it (it may happen many times along an optimization procedure) without loosing property/shape association as long as the names do not change.

[wsteffe]

Yes, klayout discretize circles into tiny edges I think that this discretization is done at the merging phase (line 46 of layout2fc.py). Yesterday I was looking inside klayout documentation searching for a way to reduce the number of points used to discretize a circle. We could also use a different library to make 2D boolean operations. I mean a library which can deal with analytic circles. But I do not know of such an alternative. Perhaps it would be easier to reconstruct the arcs after the merging is done.

[wsteffe]

I have just found this useful post regarding lost circles https://www.klayout.de/forum/discussion/1989/all-circles-changes-to-polygon

[wsteffe]

It will be much faster to just build the shape without creating object Would it possible to create directly the shape (without the sketch) and use it as the Body Base Feature ?

[wsteffe]

I have just posted a question on this forum/thread: https://www.klayout.de/forum/discussion/2087/layout2fc-a-python-script-to-export-a-3d-structure-from-klayout-work-in-progress#latest to see if the circle discretization problem can be addressed from the klayout side.