Can not create a macro that creates an object via PartDesign WB

[ceremcem]

Main Intention

It's an obvious necessity to write tests for workbenches, where I have SheetMetal WB in mind as my first objective.

My proposal is as follows:

1. Start recording a macro.
2. Create a part from scratch or load it from a file.
3. Perform relevant operations.
4. Stop the macro recording.
5. Create a Spreadsheet named Tests that will hold the expected values (lengths, angles etc.). It should contain "Description" (optional), "Expected" and "Observed" columns. Expected column should be hard coded and Observed column will be calculated by a formula.
6. Use another macro, named "Test.FCMacro", which will simply "Compare the Expected and Observed column values in the Tests Spreadsheet.

Problem

I started by recording a macro and then create a part with PartDesign WB. After recording the macro, I closed all documents and execute that macro. However, it failed with the following error:

Reproduction

1. Start recording a macro
2. PartDesign -> Create Body
3. Create sketch
4. "Pad that sketch"
5. Stop macro
6. Close document
7. Execute the saved macro.

Macro

# -*- coding: utf-8 -*-

# Macro Begin: /home/aea/.FreeCAD/Macro/a.FCMacro +++++++++++++++++++++++++++++++++++++++++++++++++
import FreeCAD
import PartDesign
import PartDesignGui
import Sketcher

# Gui.runCommand('Std_DlgMacroRecord',0)
### Begin command PartDesign_Body
App.getDocument('Unnamed').addObject('PartDesign::Body','Body')
# Gui.Selection.clearSelection()
# Gui.Selection.addSelection(App.getDocument('Unnamed').getObject('Body'))
# Gui.activateView('Gui::View3DInventor', True)
# Gui.activeView().setActiveObject('pdbody', App.getDocument('Unnamed').getObject('Body'))
App.ActiveDocument.recompute()
### End command PartDesign_Body
# Gui.Selection.addSelection('Unnamed','Body')
Gui.runCommand('PartDesign_NewSketch',0)
# Gui.Selection.clearSelection()
# Gui.Selection.addSelection('Unnamed','Body','Origin.XY_Plane.')
App.getDocument('Unnamed').getObject('Body').newObjectAt('Sketcher::SketchObject', 'Sketch', FreeCADGui.Selection.getSelection())
App.getDocument('Unnamed').getObject('Sketch').Support = (App.getDocument('Unnamed').getObject('XY_Plane'),[''])
App.getDocument('Unnamed').getObject('Sketch').MapMode = 'FlatFace'
App.ActiveDocument.recompute()
Gui.getDocument('Unnamed').setEdit(App.getDocument('Unnamed').getObject('Body'),0,'Sketch.')
# ActiveSketch = App.getDocument('Unnamed').getObject('Sketch')
# tv = Show.TempoVis(App.ActiveDocument, tag= ActiveSketch.ViewObject.TypeId)
# ActiveSketch.ViewObject.TempoVis = tv
# if ActiveSketch.ViewObject.EditingWorkbench:
#   tv.activateWorkbench(ActiveSketch.ViewObject.EditingWorkbench)
# if ActiveSketch.ViewObject.HideDependent:
#   tv.hide(tv.get_all_dependent(App.getDocument('Unnamed').getObject('Body'), 'Sketch.'))
# if ActiveSketch.ViewObject.ShowSupport:
#   tv.show([ref[0] for ref in ActiveSketch.Support if not ref[0].isDerivedFrom("PartDesign::Plane")])
# if ActiveSketch.ViewObject.ShowLinks:
#   tv.show([ref[0] for ref in ActiveSketch.ExternalGeometry])
# tv.hide(ActiveSketch.Exports)
# tv.hide(ActiveSketch)
# del(tv)
# 
# ActiveSketch = App.getDocument('Unnamed').getObject('Sketch')
# if ActiveSketch.ViewObject.RestoreCamera:
#   ActiveSketch.ViewObject.TempoVis.saveCamera()
# 
# Gui.Selection.clearSelection()
# Gui.runCommand('Sketcher_CreateRectangle',0)
geoList = []
geoList.append(Part.LineSegment(App.Vector(-18.321514,17.848700,0),App.Vector(26.595747,17.848700,0)))
geoList.append(Part.LineSegment(App.Vector(26.595747,17.848700,0),App.Vector(26.595747,-32.978725,0)))
geoList.append(Part.LineSegment(App.Vector(26.595747,-32.978725,0),App.Vector(-18.321514,-32.978725,0)))
geoList.append(Part.LineSegment(App.Vector(-18.321514,-32.978725,0),App.Vector(-18.321514,17.848700,0)))
App.getDocument('Unnamed').getObject('Sketch').addGeometry(geoList,False)
conList = []
conList.append(Sketcher.Constraint('Coincident',0,2,1,1))
conList.append(Sketcher.Constraint('Coincident',1,2,2,1))
conList.append(Sketcher.Constraint('Coincident',2,2,3,1))
conList.append(Sketcher.Constraint('Coincident',3,2,0,1))
conList.append(Sketcher.Constraint('Horizontal',0))
conList.append(Sketcher.Constraint('Horizontal',2))
conList.append(Sketcher.Constraint('Vertical',1))
conList.append(Sketcher.Constraint('Vertical',3))
App.getDocument('Unnamed').getObject('Sketch').addConstraint(conList)

# Gui.resetEdit()
App.ActiveDocument.recompute()
# ActiveSketch = App.getDocument('Unnamed').getObject('Sketch')
# tv = ActiveSketch.ViewObject.TempoVis
# if tv:
#   tv.restore()
# ActiveSketch.ViewObject.TempoVis = None
# del(tv)
# 
# Gui.Selection.addSelection('Unnamed','Body','Sketch.')
### Begin command PartDesign_Pad
App.getDocument('Unnamed').getObject('Body').newObjectAt('PartDesign::Pad','Pad', FreeCADGui.Selection.getSelection())
App.getDocument('Unnamed').getObject('Pad').Profile = App.getDocument('Unnamed').getObject('Sketch')
App.getDocument('Unnamed').getObject('Pad').Length = 10.0
App.ActiveDocument.recompute()
App.getDocument('Unnamed').getObject('Sketch').Visibility = False
App.ActiveDocument.recompute()
# App.getDocument('Unnamed').getObject('Pad').ViewObject.ShapeColor=getattr(App.getDocument('Unnamed').getObject('Body').getLinkedObject(True).ViewObject,'ShapeColor',App.getDocument('Unnamed').getObject('Pad').ViewObject.ShapeColor)
# App.getDocument('Unnamed').getObject('Pad').ViewObject.LineColor=getattr(App.getDocument('Unnamed').getObject('Body').getLinkedObject(True).ViewObject,'LineColor',App.getDocument('Unnamed').getObject('Pad').ViewObject.LineColor)
# App.getDocument('Unnamed').getObject('Pad').ViewObject.PointColor=getattr(App.getDocument('Unnamed').getObject('Body').getLinkedObject(True).ViewObject,'PointColor',App.getDocument('Unnamed').getObject('Pad').ViewObject.PointColor)
# App.getDocument('Unnamed').getObject('Pad').ViewObject.Transparency=getattr(App.getDocument('Unnamed').getObject('Body').getLinkedObject(True).ViewObject,'Transparency',App.getDocument('Unnamed').getObject('Pad').ViewObject.Transparency)
# App.getDocument('Unnamed').getObject('Pad').ViewObject.DisplayMode=getattr(App.getDocument('Unnamed').getObject('Body').getLinkedObject(True).ViewObject,'DisplayMode',App.getDocument('Unnamed').getObject('Pad').ViewObject.DisplayMode)
Gui.getDocument('Unnamed').setEdit(App.getDocument('Unnamed').getObject('Body'),0,'Pad.')
# Gui.Selection.clearSelection()
### End command PartDesign_Pad
# Gui.Selection.clearSelection()
App.getDocument('Unnamed').getObject('Pad').Length = 2.000000
App.getDocument('Unnamed').getObject('Pad').Length2 = 100.000000
App.getDocument('Unnamed').getObject('Pad').UseCustomVector = 0
App.getDocument('Unnamed').getObject('Pad').Direction = (1, 1, 1)
App.getDocument('Unnamed').getObject('Pad').Type = 0
App.getDocument('Unnamed').getObject('Pad').UpToFace = None
App.getDocument('Unnamed').getObject('Pad').Reversed = 0
App.getDocument('Unnamed').getObject('Pad').Midplane = 0
App.getDocument('Unnamed').getObject('Pad').Offset = 0
App.getDocument('Unnamed').recompute()
# Gui.getDocument('Unnamed').resetEdit()
# Macro End: /home/aea/.FreeCAD/Macro/a.FCMacro +++++++++++++++++++++++++++++++++++++++++++++++++

Version

OS: Debian GNU/Linux 10 (buster)
Word size of OS: 64-bit
Word size of FreeCAD: 64-bit
Version: 0.19.26091 (Git)
Build type: Release
Branch: LinkStage3
Hash: eb10f4011b36086ed70e8f667c70c10c6d2ecb3e
Python version: 3.7.3
Qt version: 5.11.3
Coin version: 4.0.0a
OCC version: 7.3.0
Locale: English/UnitedStates (en_US)

[realthunder]

That's a tough one. Currently, many PartDesign commands do not really use Python to perform its task. For some that do use Python, they are often not fully done in Python. This situation is not limited to PartDesign. Even though asm3 is pure Python workbench, most of its command is not logged in Python console.

So in order to achieve what you proposed here, we will have to first make sure that all commands must be fully repeatable by a recorded macro. The problem is, even if you can do that, it is often not enough to detect any potential problem by just compare the log output. For example, the resulting shape maybe wrong, and there is no easy way to compare shape

[ceremcem]

For example, the resulting shape maybe wrong, and there is no easy way to compare shape

I didn't think about that. Isn't it enough to compare the output XML files, tag by tag and value by value?

Currently, many PartDesign commands do not really use Python to perform its task. For some that do use Python, they are often not fully done in Python. This situation is not limited to PartDesign. Even though asm3 is pure Python workbench, most of its command is not logged in Python console.

Do you mean that an operation is performed by directly calling a C/C++ function within that WB, without using any Python wrapper?

If so, is it (theoretically) possible to write wrappers (or edit existing ones if necessary) for those "directly called" functions and "print" their Python equivalents into the Python console, so we wouldn't touch any WB's code but we could still repeat the exact same operations via Python API? If yes, is it a logical approach for such a problem?

[realthunder]

I didn't think about that. Isn't it enough to compare the output XML files, tag by tag and value by value?

OCC use an opaque format to store BRepShape. Although it can be stored in text format, it is not meant to be processed by external program. I am not sure if their file contains any random value. It is possible that we can compare the text file if they are produced by the exact same OCC version. On the other hand, my topo naming encoding does contain random value, to reduce possible name clash for shapes from different documents. The root of this randomness is the object ID. each object in a document is assign a monotonically increasing ID, but the start of the ID value is random. The ID is encoded into topo name for history tracing. I can of course add new Python API to let user specify starting ID.

Do you mean that an operation is performed by directly calling a C/C++ function within that WB, without using any Python wrapper?

The command logging in FreeCAD Python console is a service provided by FreeCAD, not Python. In order to do that, the workbench code must explicitly construct its python code in text, and then calls FreeCADGui.doCommand() to run the code. And they must do this for every step that are meant to be logged by Python console. It's an extra step and quite tedious, which is why some workbench is not doing that. But it is definitely beneficial doing that. I'll make change to PartDesign commands when it gets more stable.

[realthunder]

Forgot to mention one thing, it is probably more convenient to directly invoke the command through Gui.runCommand() rather than the code captured by macro recording.

The logic of macro recording is like this. If the workbench command does not log any command through doCommand(), the macro recorder will log the command invoking as something like Gui.runCommand('asm3CmdNew',0). This is the code to invoke the command in its entirety, and has a higher chance to repeat correctly.

If the workbench command wants to log command by itself through doCommand(), then the macro recorder will put some special comments to mark the beginning and ending of the logged command like this,

### Begin command PartDesign_Body
App.getDocument('Unnamed').addObject('PartDesign::Body','Body')
App.getDocument('Unnamed').getObject('Part').addObject(App.getDocument('Unnamed').getObject('Body'))
import PartDesignGui
Gui.Selection.clearSelection()
Gui.Selection.addSelection(App.getDocument('Unnamed').getObject('Part'), 'Body.')
Gui.activateView('Gui::View3DInventor', True)
Gui.activeView().setActiveObject('pdbody', App.getDocument('Unnamed').getObject('Body'))
App.ActiveDocument.recompute()
### End command PartDesign_Body

Macros recorded this way will rely on the workbench code to actually log every python code it runs to make sure it can be repeated, and sometimes this is not the case. From the special comment, you can find out the command name, and directly invoke the command in its entirety as Gui.runCommand('PartDesign_Body').

[ceremcem]

I can of course add new Python API to let user specify starting ID.

That would perfectly resolve that "randomness" issue on recreation, as the rest of the id's are deterministic.

the workbench code must explicitly construct its python code in text, and then calls FreeCADGui.doCommand() to run the code ... I'll make change to PartDesign commands when it gets more stable.

As you mentioned, Gui.runCommand() would suffice at this moment. (My SheetMetalUnfoldUpdater.FCMacro depends on that, for example.)

I also want to mention that this "recording the PartDesign WB macro and re-running it" works OK in upstream.

Importance

I think this will be a killer feature because it will instantly turn our new projects into tests cases (I propose an option like "auto record logs (recreation macros)"), which will in turn eliminate the huge problem in front of using FreeCAD in the industry: Breaking changes in FreeCAD introduce huge costs.

This "auto create test cases" will also let us quickly clear/improve our workbenches without worrying about breaking anything.

[realthunder]

This "auto create test cases" will also let us quickly clear/improve our workbenches without worrying about breaking anything.

I doubt if it will work as you have intended. I can't say much about OCC BRep format, but for my topo naming, the internals keeps changing, as new bugs are fixed and algorithm improved. I am about to push out a big change that completely changes its internal format for much improved efficiency. The irony is that these changes, if work correctly, will be transparent to end-user, but will inevitably break the very test that is intended to catch breaking change.

[ceremcem]

I am about to push out a big change that completely changes its internal format for much improved efficiency.

Does it mean that those changes will break our previous models?

[realthunder]

No. I am trying very hard to make sure my future release can read both the old and new format, which is why it takes quite some time. I've been working on this on and off for several months already. Even the older release can load newer file, but it won't recognize it, and will prompt the user for full recompute, just like when you load file saved using upstream FC. However, as you know, a full recompute has a higher chance of error, which leads back to the purpose of your post here.

All in all, it's a difficult problem to solve, especially for a program with continuous releasing scheme, and with many external dependencies which are continuous releasing by themselves. You know that many model break on full recompute is due to OCC version change.

[ceremcem]

No. I am trying very hard to make sure my future release can read both the old and new format

That's great news. I have a pretty good idea about how hard to stay compatible with old version is, so, I appreciate it.

However, "Whatever can happen will happen".

Python advances from version 2 to version 3 and our code/tools get broken. OCC advances, our work gets broken. Any day something may be upgraded and our previous works may get broken. This will happen some day, we can't prevent this from happening. Remember FC 0.16 to 0.17 transition. FreeCAD is a complex tool like the Space Shuttle. Anything may go wrong at any time.

What we can only do is to get prepared for that day.

This entire conversation/idea didn't come from nowhere. I recently wanted to port my previous work (made with LinkStage3@39c32719) to current LinkStage3 release. I'm confused for a while about where to start. There is a huge work to do. When we try to use Python2 script with Python3 interpreter, it throws meaningful error messages which guides us while porting. If we have a test suite, then we are good to go. There is no such a facility here.

How can I test the project when I'm done porting? There is only a solution I can think of: I'll re-generate my production data (laser cuts, bend calculations, etc.) that will match with the previous output. How do I start drawing the model in the first place? I need to redo everything I see in the object tree.

Then I realized that the "verbal definition of the model" (= object tree) is not changing. This is kind of "logs". If I had such an explicit definition of my model, it will be a trivial work to redo everything:

When our test cases get broken, we can "upgrade" our "logs" that will generate the correct "outputs". If we catch the breaking change as early as possible, it will be way easier to write a small "upgrade procedure" (a patch for the logs) and share them.

As this is my personal way of thinking and as I believe I expressed the underlying idea clearly, it's up to you to decide, so I'm going to close this issue.

[realthunder]

I think for this kind of task with the given complexity of the software and dependencies, human intervention cannot be spared, at least for the foreseeable future. I do intend to work on a shape diff tool later to make it easier for visual inspection.