Generalize Exposing Properties

[pieterhijma]

Proposal description

The essence of the proposal is generalizing exposing properties as some have suggested (for example here and here). By changing FreeCAD's execution model slightly, it may be possible to remove the reliance on temporary hidden files as is the case for SubShapeBinders, something that makes FreeCAD models brittle (in my humble opinion). It will provide a Part::Variant document object that does not rely on hidden temporary files, making use of FreeCAD's tweaked execution model.

The proposed changes are adding an "Exposed" flag to properties similar to "Copy on Change", separating computing a shape from setting a shape in the execute phase of FreeCAD's execution model, and introducing a Part::Variant that makes use of this functionality.

This will provide FreeCAD users with something that I call an Application Geometry Interface (see #12120) something that is to the best of my knowledge not possible in other CAD programs (other than coded CAD solutions). Instead of relying on temporary hidden files, this proposed solution would be fully supported by the FreeCAD core (not only aligning with objective Streamlined Workflow, but also with objective Prevent shooting yourself in the foot). This conversation explains the shortcomings of Spreadsheet configuration tables well and in my humble opinion the same is true for variants based on SubShapeBinders.

Deliverables

• PR Generalize execution model: In this PR we investigate changing the execution model to separate setting a shape and computing a shape. I have already started a discussion for this (#12868).
• PR Expose properties: In this PR we investigate to what extent we can "call" computing shapes with properties that are exposed.
• PR Part::Variant: In this PR we investigate to what extent we can create variants in a similar way to SubShapeBinders without the requirement to have temporary documents.
• PR Tests or discussion: Extensive tests for the implemented behavior or a report in the PR or issue discussing problems.

These are low level changes that make temporary hidden documents and copying of object structure obsolete, providing a more fundamental solution to variants and configurations. For users, it means that they can expose properties to achieve a similarly intuitive interface to document objects in general as the Part primitives such as Cube and Cylinder provide (having a property such as radius that changes the shape).

Timeline

• Generalize execution model: Start in May, finish end of May
• Expose properties: Start in June, finish end of June
• Part::Variant: Start in July, finish end of August
• Tests or discussion: Start in September, finish end of September

Risks and mitigation

I would like to carry out this work as an independent contractor.

The above sketched project has inherent risk because of a change to the execution model, albeit a small change. However, this change will be threaded throughout FreeCAD, so potentially many document objects are touched. I will put effort into making the changes as backward compatible as possible. However, because the potential of this functionality is so high, I think it is at least worth of investigating the proposed changes. Since this is quite a fundamental change, it is difficult to promise that the PRs will be ready to be merged after each stage, but without a doubt, it will provide us with a better understanding of the problems at hand to support the above functionality. Because of this inherent risk, the last stage is called Tests or discussion that allows us to get a better understanding in case unforeseen issues arise that prevent generalizing exposed properties.

Two challenges stand out at this moment:

• dependency management, and (slightly related)
• cyclic dependencies

It is at this time not clear how to properly manage dependencies. That is, if an exposed property is changed, it should trigger a recompute of the object. This is most likely doable, but at this stage it is unclear to me how to trigger the right recompute and how to prevent large expensive recomputes.

Exposing properties will depend on properties of parent document objects. Currently, this is not allowed in FreeCAD because of cyclic dependencies, so it is necessary to find a solution for the proposed behavior, possible by making informed decisions on where to allow a cyclic dependency.

Compensation

I would like to carry out this work with for a total of 8000 EUR.

About you

My name is Pieter Hijma (pieterhijma on the forum and pieterhijma on GitHub). I'm an independent contractor (https://pieterhijma.net) and as a co-founder of the Open Toolchain Foundation, I would love to contribute to improve FreeCAD as it is an important part of open toolchains. This functionality is inspired by my personal issues regarding modularity as a FreeCAD user. For example, my post on my Fab Academy program already mentions using different categories for parameters.

I have been working on Variable Sets as an independent contractor for Ondsel. This work can be regarded as an extension of that work, but with a more fundamental solution that was out of the scope of the Ondsel contract. In that sense, I have experience contributing to FreeCAD and experience relevant to this proposed work.

I haven't worked on similar software before but I have solved similar tasks that are more architectural of nature before. An example is the Cashmere project ([1], [2]) that needed a clean interface between two existing systems.

[chennes]

Thank you for the proposal, @pieterhijma. The Grant Review Committee has begun evaluating these proposals, and requests a higher-level explanation of your proposal. What will users gain from this work?

[pieterhijma]

Thank you for this opportunity, because indeed, the issue template did not allow a more high-level description.

High-level description

When opening freecad.org the description of FreeCAD is "Your own 3D parametric modeler". Although FreeCAD is a fantastic program, I would argue that FreeCAD has limitations in parametric design despite having a fantastic structure with document objects and properties.

Arguably, the most intuitive objects in FreeCAD are the Part Cube, Cylinder, Sphere, and related geometry. What makes it intuitive is that all objects have properties that relate to the geometry, such as length, width, and height for the cube, and radius and height for the cylinder. Changing one of those properties changes the shape of the object.

This is what I would call a good example of a proper Application Geometry Interface. For the user it is clear how to parameterize a cube, cylinder, and sphere, namely by simply changing those properties. This gives users a great amount of flexibility: it is possible to create a design with several variants of that geometry, all with different parameters, for example a design with different cylinders with different radii.

The objects in Part are highly modular: the user simply creates a new cylinder and drives the radius from "outside" the geometry by simply setting a property. Having this modularity, allows a user to exchange an object such as a cylinder with other users that can parameterize the object themselves by setting the properties to a value they prefer. Having this kind of modularity also stimulates reuse of geometry. Users can easily reuse such geometry in different designs, tailored to their needs by means of setting properties.

However, this intuitive interface to document objects for variants is not available in general in FreeCAD. For example, geometry designed in Part Design cannot provide the user with such an interface because it would require using parameters that would need to be defined in a parent object such as Body or Part. This creates cyclic dependencies that are not allowed in FreeCAD.

In the Part workbench itself, the intuitive interface disappears as soon as you combine the predefined geometry such as Cube and Cylinder. For example, users cannot fuse a cube and a cylinder and create properties in the fuse object that drive the length and radius of the fused cube and cylinder.

Variable sets attempt to introduce this kind of interface in a limited way. For now it is necessary to use variable sets and it is only available in App::Part. However, some have already suggested to generalize this behavior, for example here and here.

Variable sets promise the ability to parameterize geometry from outside the context of the part, which is the key for modularity and reuse in design. This is very powerful and to the best of my knowledge not possible in other CAD programs (other than coded CAD solutions). Together with SubShapeBinders, variable sets can create relatively low-cost variants of which the design can be modified affecting all variants (unlike clones or copies).

In this comment I show a video that explains the modularity problems for a simple box. Then in this comment I show a video with the same box designed with variables sets. The panel and grooved panel are modified from the outside without affecting the original geometry, providing modularity and reuse.

Currently, this functionality has limitations and is still not generically available in FreeCAD because 1) it needs Variable Sets, 2) it only works with geometry enclosed in Part, and 3) it is implemented in terms of hidden references and SubShapeBinders that create hidden temporary files. This is not ideal and makes the functionality brittle. In the proposed work, I would like to investigate to acquire this functionality in general in FreeCAD and with an improved implementation.

If implemented fully, what does this functionality provide FreeCAD users? Suppose there are users that are designing objects: They can design geometry with Part Design, Part or any other workbench. They can think about how this geometry should be parameterized by people that want to use the design. They create properties such as length, width, height in the top-level document object of the design and mark these properties as "Exposed". Doing so, they create a contract with the people who use this geometry.

Suppose there are users that create designs by using geometry provided by someone else: They obtain the geometry and find in the top-level document object properties that are marked "Exposed". They can then link to this geometry once or multiple times and for each time they can override the exposed properties with their own values, creating variants in their design without (a crucial difference with the current situation) affecting the original design.

Related work

Some may argue that FreeCAD has the above described functionality and to some extent this is true. However, in my humble opinion they don't provide an Application Geometry Interface as outlined above and the implementations have drawbacks.

• Copy-on-change links: This does not provide the user copy-on-change properties in the top-level document object and as such to acquire the above functionality, users would need to create links on different levels in geometry, making it a very complicated solution. The implementation also makes a full copy of the linked geometry. To acquire properties in the top-level document object, designers would have to make use of hidden references, making the functionality difficult to use for users that are not programmers and fully understand the dangers of cyclic dependencies. Moreover, if the original design changes, then the copy-on-change links don't receive these updates.
• Copy-on-change links with tracking: The latter problem is solved with tracking, but it still makes a complete copy of the geometry and it still requires hidden references for acquiring top-level properties.
• SubShapeBinders: The current exposed variable sets make use of SubShapeBinders. Instead of storing a full copy of the geometry, the SubShapeBinder only stores the computed shape. The shape is computed by means of copying the original geometry into a temporary hidden file, apply the copy-on-change changes to recompute the shape, and then use that shape in the SubShapeBinder. The hidden temporary file is not as hidden as it should be (for example it is visible in the external link dialog), and dependency tracking is very challenging, because the SubShapeBinder depends on the object in the temporary hidden file, and the object in the temporary hidden file depends on the original object. This allows for many opportunities for dependency mistakes, resulting in geometry that is not updated correctly.
• Assembly 4 Variants: See this discussion for more information. A drawback of this implementation is that it only works with Assembly 4. It also makes use of temporary hidden documents making the variants of limited use because it is not possible to attach the variant to other parts in the assembly, see this discussion.
• Spreadsheet Configurations: Spreadsheet configurations come close to an Application Geometry Interface by creating an enumeration property in the top-level document object that drives configurations of parameters set by the designer of the object. However, a drawback of these configuration compared to the proposed solution is that the parameterization is limited to predefined configurations. See the wiki and forum for explanation. It makes use of hidden references and it is not very flexible. Additionally, the implementation is very brittle in my humble opinion, see this comment from a user that has experience with configuration tables. I fully agree with the comments and in my humble opinion it is also illustrative for the drawbacks with hidden references and temporary hidden files.

Since there are many attempts to acquire variants, configurations, and related concepts, but none are satisfying including my own version of variable sets, I hope that the FreeCAD community, in this case represented by the FPA, would allow me to research this very important feature of CAD that, to the best of my knowledge, could also set FreeCAD apart from other CAD solutions.

Context

The fact that FreeCAD is free software makes it an excellent choice for exchanging designs because the file format is open. Since FreeCAD's goal is to be a parametric modeler, I would like FreeCAD to fully follow through by also allowing exchanging parametric designs. I would hope these designs contain a well-defined interface to users, expressing design intent from designers and allowing users of designs to tailor the design to their need as defined by the designers by means of an Application Geometry Interface.

It is highly likely that exchanging designs becomes more and more important in the future. I've participated in the EU-funded INTERFACER project that aimed to create a digital infrastructure to allow exchanging designs to produce goods locally. This is related to the Fab City Global Initiative that sets the very ambitious goal to produce all goods in a city locally, recognizing that sending bits over the globe is cheap, whereas sending atoms (shipping) is very expensive. The Internet of Production Alliance is another organization that aims for a similar future.

This proposal is partly motivated by this context because I can foresee FreeCAD to be an important driver for this future, especially if parametric design becomes exchangeable as I outlined above. Another important motivation for this work is that it simply makes parametric design easier to manage for users because it allows for more parameterization and it doesn't affect the original geometry.

I hope this helps the committee to make an informed decision. Please let me know if more information is required.

[shaise]

Hi Pieter,

Thank you for your elaborate explanation, I now understand your proposal much better. In my opinion, having a top level parameter object that is dedicated for this purpose (as opposed to spreadsheet, which is more like a workaround) is a great idea. This object should be easy to create and populate. The spreadsheet option is cumbersome and it will be nice if something simpler is implemented. I have two questions:

1. Can you have multiple variableSet objects in the same document?
2. Can this Variable set be inside a Part container, so if you make a linked copy of this part in the same document, you can give it other values
   • making part variants in the same document? Thanks!

shai

On Wed, Apr 3, 2024 at 2:35 PM Pieter Hijma @.***> wrote:

Thank you for this opportunity, because indeed, the issue template did not allow a more high-level description. High-level description

When opening freecad.org https://www.freecad.org/ the description of FreeCAD is "Your own 3D parametric modeler". Although FreeCAD is a fantastic program, I would argue that FreeCAD has limitations in parametric design despite having a fantastic structure with document objects and properties.

Arguably, the most intuitive objects in FreeCAD are the Part Cube, Cylinder, Sphere, and related geometry. What makes it intuitive is that all objects have properties that relate to the geometry, such as length, width, and height for the cube, and radius and height for the cylinder. Changing one of those properties changes the shape of the object.

This is what I would call a good example of a proper Application Geometry Interface. For the user it is clear how to parameterize a cube, cylinder, and sphere, namely by simply changing those properties. This gives users a great amount of flexibility: it is possible to create a design with several variants of that geometry, all with different parameters, for example a design with different cylinders with different radii.

The objects in Part are highly modular: the user simply creates a new cylinder and drives the radius from "outside" the geometry by simply setting a property. Having this modularity, allows a user to exchange an object such as a cylinder with other users that can parameterize the object themselves by setting the properties to a value they prefer. Having this kind of modularity also stimulates reuse of geometry. Users can easily reuse such geometry in different designs, tailored to their needs by means of setting properties.

However, this intuitive interface to document objects for variants is not available in general in FreeCAD. For example, geometry designed in Part Design cannot provide the user with such an interface because it would require using parameters that would need to be defined in a parent object such as Body or Part. This creates cyclic dependencies that are not allowed in FreeCAD.

In the Part workbench itself, the intuitive interface disappears as soon as you combine the predefined geometry such as Cube and Cylinder. For example, users cannot fuse a cube and a cylinder and create properties in the fuse object that drive the length and radius of the fused cube and cylinder.

Variable sets https://github.com/FreeCAD/FreeCAD/pull/12532 attempt to introduce this kind of interface in a limited way. For now it is necessary to use variable sets and it is only available in App::Part. However, some have already suggested to generalize this behavior, for example here https://github.com/FreeCAD/FreeCAD/pull/12135#issuecomment-1913340757 and here https://github.com/FreeCAD/FreeCAD/pull/12135#issuecomment-1952882504.

Variable sets promise the ability to parameterize geometry from outside the context of the part, which is the key for modularity and reuse in design. This is very powerful and to the best of my knowledge not possible in other CAD programs (other than coded CAD solutions). Together with SubShapeBinders, variable sets can create relatively low-cost variants of which the design can be modified affecting all variants (unlike clones or copies).

In this comment https://github.com/FreeCAD/FreeCAD/issues/12120#issuecomment-1956487505 I show a video that explains the modularity problems for a simple box. Then in this comment https://github.com/FreeCAD/FreeCAD/pull/12532#issuecomment-1956491941 I show a video with the same box designed with variables sets. The panel and grooved panel are modified from the outside without affecting the original geometry, providing modularity and reuse.

Currently, this functionality has limitations and is still not generically available in FreeCAD because 1) it needs Variable Sets, 2) it only works with geometry enclosed in Part, and 3) it is implemented in terms of hidden references and SubShapeBinders that create hidden temporary files. This is not ideal and makes the functionality brittle. In the proposed work, I would like to investigate to acquire this functionality in general in FreeCAD and with an improved implementation.

If implemented fully, what does this functionality provide FreeCAD users? Suppose there are users that are designing objects: They can design geometry with Part Design, Part or any other workbench. They can think about how this geometry should be parameterized by people that want to use the design. They create properties such as length, width, height in the top-level document object of the design and mark these properties as "Exposed". Doing so, they create a contract with the people who use this geometry.

Suppose there are users that create designs by using geometry provided by someone else: They obtain the geometry and find in the top-level document object properties that are marked "Exposed". They can then link to this geometry once or multiple times and for each time they can override the exposed properties with their own values, creating variants in their design without (a crucial difference with the current situation) affecting the original design. Related work

Some may argue that FreeCAD has the above described functionality and to some extent this is true. However, in my humble opinion they don't provide an Application Geometry Interface as outlined above and the implementations have drawbacks.

• Copy-on-change links: This does not provide the user copy-on-change properties in the top-level document object and as such to acquire the above functionality, users would need to create links on different levels in geometry, making it a very complicated solution. The implementation also makes a full copy of the linked geometry. To acquire properties in the top-level document object, designers would have to make use of hidden references, making the functionality difficult to use for users that are not programmers and fully understand the dangers of cyclic dependencies. Moreover, if the original design changes, then the copy-on-change links don't receive these updates.
• Copy-on-change links with tracking: The latter problem is solved with tracking, but it still makes a complete copy of the geometry and it still requires hidden references for acquiring top-level properties.
• SubShapeBinders: The current exposed variable sets make use of SubShapeBinders. Instead of storing a full copy of the geometry, the SubShapeBinder only stores the computed shape. The shape is computed by means of copying the original geometry into a temporary hidden file, apply the copy-on-change changes to recompute the shape, and then use that shape in the SubShapeBinder. The hidden temporary file is not as hidden as it should be (for example it is visible in the external link dialog), and dependency tracking is very challenging, because the SubShapeBinder depends on the object in the temporary hidden file, and the object in the temporary hidden file depends on the original object. This allows for many opportunities for dependency mistakes, resulting in geometry that is not updated correctly.
• Assembly 4 Variants: See this discussion https://forum.freecad.org/viewtopic.php?t=62767 for more information. A drawback of this implementation is that it only works with Assembly 4. It also makes use of temporary hidden documents making the variants of limited use because it is not possible to attach the variant to other parts in the assembly, see this discussion https://forum.freecad.org/viewtopic.php?p=623033#p623033.
• Spreadsheet Configurations: Spreadsheet configurations come close to an Application Geometry Interface by creating an enumeration property in the top-level document object that drives configurations of parameters set by the designer of the object. However, a drawback of these configuration compared to the proposed solution is that the parameterization is limited to predefined configurations. See the wiki https://wiki.freecad.org/Spreadsheet_Workbench#Configuration_tables and forum https://forum.freecad.org/viewtopic.php?f=17&t=42183 for explanation. It makes use of hidden references and it is not very flexible. Additionally, the implementation is very brittle in my humble opinion, see this comment https://github.com/FreeCAD/FreeCAD/pull/11971#issuecomment-1913716004 from a user that has experience with configuration tables. I fully agree with the comments and in my humble opinion it is also illustrative for the drawbacks with hidden references and temporary hidden files.

Since there are many attempts to acquire variants, configurations, and related concepts, but none are satisfying including my own version of variable sets, I hope that the FreeCAD community, in this case represented by the FPA, would allow me to research this very important feature of CAD that, to the best of my knowledge, could also set FreeCAD apart from other CAD solutions. Context

The fact that FreeCAD is free software makes it an excellent choice for exchanging designs because the file format is open. Since FreeCAD's goal is to be a parametric modeler, I would like FreeCAD to fully follow through by also allowing exchanging parametric designs. I would hope these designs contain a well-defined interface to users, expressing design intent from designers and allowing users of designs to tailor the design to their need as defined by the designers by means of an Application Geometry Interface.

It is highly likely that exchanging designs becomes more and more important in the future. I've participated in the EU-funded INTERFACER project https://www.interfacerproject.eu/ that aimed to create a digital infrastructure to allow exchanging designs to produce goods locally. This is related to the Fab City Global Initiative https://fab.city/ that sets the very ambitious goal to produce all goods in a city locally, recognizing that sending bits over the globe is cheap, whereas sending atoms (shipping) is very expensive. The Internet of Production Alliance https://www.internetofproduction.org/ is another organization that aims for a similar future.

This proposal is partly motivated by this context because I can foresee FreeCAD to be an important driver for this future, especially if parametric design becomes exchangeable as I outlined above. Another important motivation for this work is that it simply makes parametric design easier to manage for users because it allows for more parameterization and it doesn't affect the original geometry.

I hope this helps the committee to make an informed decision. Please let me know if more information is required.

— Reply to this email directly, view it on GitHub https://github.com/FreeCAD/FPA-grant-proposals/issues/3#issuecomment-2034337131, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMLKQE6O4H6ALKTBR3N6GLY3PSPNAVCNFSM6AAAAABFM54LJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZUGMZTOMJTGE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[pieterhijma]

Hi @shaise, thank you for your interest and great to hear it helps.

The answer to question 1: Yes, no problem at all. I would say this would be very common. The second video in this comment also shows this in the assembly file.

The answer to question 2: The Variable Set can be inside a Part container, this is already possible in the current implementation. Making part variants in the same document is definitely possible. Giving it other values happens in a bit different way: You create a copy of the original variable set and then in the top-level document object, you indicate that this variable set is the one that drives the variant of the part. Changing the parameters of the latter variable set changes the variant. So in effect, the original variable set is overridden by the one you selected. By the way, it is also possible to have multiple variable sets inside the part that represent different configurations, for example small, medium, large. The user can then select one of those to override the original variable set. So, in this way, the designer of an object can also provide the user with predefined configurations.

[shaise]

Sounds great! Thanks!

On Wed, Apr 3, 2024 at 3:34 PM Pieter Hijma @.***> wrote:

Hi @shaise https://github.com/shaise, thank you for your interest and great to hear it helps.

The answer to question 1: Yes, no problem at all. I would say this would be very common. The second video in this comment https://github.com/FreeCAD/FreeCAD/pull/12532#issuecomment-1956491941 also shows this in the assembly file.

The answer to question 2: The Variable Set can be inside a Part container, this is already possible in the current implementation. Making part variants in the same document is definitely possible. Giving it other values happens in a bit different way: You create a copy of the original variable set and then in the top-level document object, you indicate that this variable set is the one that drives the variant of the part. Changing the parameters of the latter variable set changes the variant. So in effect, the original variable set is overridden by the one you selected. By the way, it is also possible to have multiple variable sets inside the part that represent different configurations, for example small, medium, large. The user can then select one of those to override the original variable set. So, in this way, the designer of an object can also provide the user with predefined configurations.

— Reply to this email directly, view it on GitHub https://github.com/FreeCAD/FPA-grant-proposals/issues/3#issuecomment-2034487753, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMLKQHRJNDOXSSTYRZCTY3Y3PZPBAVCNFSM6AAAAABFM54LJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZUGQ4DONZVGM . You are receiving this because you were mentioned.Message ID: @.***>

[chrisb-github]

I thin kI understand (partially) the meaning of such a variant object. As I'm not familiar with variants (neither with FreeCAD's configuration tables, nor with other systems) there remain some questions:

• It is not clear to me how a user would switch between different variants
• Is this concept similar to objects from the Dynamic Data workbench implemented in core?
• Would your implementation overcome the dependency cycles, which are currently calculated on the object level?

[pieterhijma]

Thanks for asking more information.

I thin kI understand (partially) the meaning of such a variant object. As I'm not familiar with variants (neither with FreeCAD's configuration tables, nor with other systems) there remain some questions:

• It is not clear to me how a user would switch between different variants

The term variant is a bit overloaded, because some think of a variant in terms of fixed configurations, for example a design of an m3, m8, or m10 nut where you have the same design, but there is an enumeration property which allows you to choose between those three variants. I'm thinking of variants as parameterized objects that may have these kinds of fixed configurations, but allow you to set parameters in general if the designer has decided to do so by marking top-level properties as "Exposed". Creating a variant would entail nothing more than creating an App::Variant object based on the design and setting one of the exposed properties.

I think it may be useful to give an example: Suppose a designer designs an aluminum profile. The profile is sketched in Sketcher, it is padded to a certain length and wrapped in an App::Part. The designer decides that users of this design can set the size of the profile, for example 20, 30, or 40 mm, and the length. The App::Part will have the exposed properties Size and Length and the Sketch and Pad make use of that (circumventing cyclic dependency problems), so setting these values in an App::Variant will create a new variant. So, the tree looks like this:

App::Part MyProfile
  Body
    Pad
      Sketch (of the profile)

App::Part MyProfile has two exposed properties Size and Length.

Now, there is someone that wants to use these designs to create a fancy box. It needs aluminum profiles of size 20x20 and 30x30 and with different lengths. In the current FreeCAD version, we might copy in MyProfile a couple of times and adjust the value in Pad and Sketch and assemble with that. Suppose we do that, but we notice that the Sketch misses some details that are necessary for the box. Unfortunately, if we adjust the Sketch, these changes are not transferred to any of the profiles used in the assembly because they are copies. We have to somehow copy the new Sketch in all places where the object is used.

With the proposed changes, the user would create App::Variants from App::Part MyProfile. Because this App::Part exposes properties Size and Length, the App::Variant will also obtain properties Length and Size and setting these values leads to a new variant. With this we create the various variants in the assembly with the values we would like, different lengths, different sizes. We then come to the realization that the Sketch misses some details. We change the Sketch and all changes are automatically transferred to the assembly, because the profiles used in the assembly are all based on the original design.

Since the variants automatically obtain the properties Length and Size, the interface to adjusting it to your needs is exactly the same as for Part Box, Cylinder and similar predefined objects. In a sense, users of FreeCAD will be able to design their own primitive objects such as Box, Cylinder with properties they find meaningful.

• Is this concept similar to objects from the Dynamic Data workbench implemented in core?

Variable sets are similar to objects from the Dynamic Data workbench and they are implemented in core, because it allows it to behave in a special way for the parameterization discussed above. If you make a variable set exposed, then you can set a whole "set of variables" in one go by changing a link to an equivalent variable set. Dynamic Data allows you to make configurations (only predefined values as discussed above, similar to what spreadsheet configuration tables do). So, variable sets are similar but more powerful. This concept generalizes variable sets to allow properties to be exposed in general. It is more similar to copy-on-change properties but with a different implementation trying to get a more robust implementation by changing the execution model slightly.

• Would your implementation overcome the dependency cycles, which are currently calculated on the object level?

This is a very good question and handling dependency cycles will be a challenge. I don't know if dependency cycles can be overcome nor that it is good to do so. FreeCAD is designed with a DAG in mind, a directed acyclic graph. This is a good choice because it is always clear what the dependencies are. However, I believe two patterns emerge regarding cyclic dependencies:

• I believe it occurs that users are sometimes unknowingly creating a dependency cycle and I also think it is not completely obvious to most users how to solve this and find a good alternative to what they want.
• In addition, another pattern is that more powerful features are harmed by the restrictions on cyclic dependencies. For configuration tables in spreadsheets, hiddenref was introduced precisely to be hidden from cyclic dependency checks.

The proposed changes where properties are present in parent objects that are referred to in child objects have the same problem, so I will certainly investigate potential solutions. I believe in your question you are referring to the granularity of recomputing objects. Finding a more fine-grained solution is certainly a potential direction, but I could also foresee to move to a more fixed-point kind of computation or somehow excluding exposed properties from the cyclic dependency check if it is safe to do so.

I hope this makes the work more clear. Please let me know if you need more information.

[shaise]

Hi @pieterhijma ,

Where exactly do you "Expose" this parameters? Do you make a new "parameters object", or do you just mark the "Length" parameter of the Pad object as exposed?

In my vision I see the following:

App::Part MyProfile
  Parameters
  Body
    Pad
      Sketch (of the profile)

Parameters will contain any parameter a want with a name and a value. I will then use these parameters in the sketch and Pad of the Body.

Now, the variant is a simple link to MyProfile. in this link everything is the same as any other link, meaning that any changes to the original part will reflect here as well. Only Parameters are a copy of the original parameters, hence you can change them here and it will essentially create a variant. (same as the origin is a copy of the original origin, but now you can change it and put the part in another location) "Parameters" object can also be at top level of the document if you want parameters to be used across multiple parts.

shai

[yorikvanhavre]

If I understand correctly, your variable set is basically a property container that has the ability to change properties of its children, thus bypassing the cyclic dependency limitation, is that correct?

[pieterhijma]

Thank you all for the great questions!

Where exactly do you "Expose" this parameters? Do you make a new "parameters object", or do you just mark the "Length" parameter of the Pad object as exposed?

Both are possible: you can mark properties as Exposed and you can create a variable set and make that one exposed. The benefit of the former is that you can expose properties in geometry that doesn't allow to contain variable sets, such as Body (hence the title of the proposal to generalize exposing properties). The benefit of the latter is that you can expose multiple properties at once and override them at once.

In my vision I see the following:

App::Part MyProfile
  Parameters
  Body
    Pad
      Sketch (of the profile)

Parameters will contain any parameter a want with a name and a value. I will then use these parameters in the sketch and Pad of the Body.

Now, the variant is a simple link to MyProfile. in this link everything is the same as any other link, meaning that any changes to the original part will reflect here as well. Only Parameters are a copy of the original parameters, hence you can change them here and it will essentially create a variant. (same as the origin is a copy of the original origin, but now you can change it and put the part in another location) "Parameters" object can also be at top level of the document if you want parameters to be used across multiple parts.

Yes, this would be the latter scenario that I discussed above. This could be a possibility but the current implementation in #12532 works a bit differently to achieve the same kind of interface that objects from the Part workbench provide. The main idea is that you leave everything in the variant alone and manipulate the parameters from the outside, sort of injecting new parameters and overriding them automatically. In the implementation in #12532, there is a link property that points to the object "Parameters". Then you can set that link to an equivalent object "Parameters" with the new values. The benefit is that the interface to creating variants is more consistent and nothing inside needs to be touched.

It may be useful to compare with programming in which you only have functions that don't accept parameters:

f() {
  a = 2
  b = 3
  return a + b
}

print(f()) # prints 5

Now to compute the sum of 3 and 4, you need to copy f() and change a and b to 3 and 4. You can compare this way of programming to what is the approach in FreeCAD currently but then for geometry. It would be useful to be able to inject parameters from outside without affecting the function f():

f(a, b) {
  return a + b
}

print(f(2, 3)) # prints 5
print(f(3, 4)) # prints 7

Note that in this pseudo code, the values 2, 3 and 3, 4, come from the outside and are somehow injected into f() where a and b assume the values from outside. So, in FreeCAD to acquire properties that you can manipulate from the outside without affecting the geometry, I propose to make an Exposed flag to properties. Working in the latter scenario above with a variable set, you can compare that with a struct or dictionary that you pass to geometry.

[pieterhijma]

If I understand correctly, your variable set is basically a property container that has the ability to change properties of its children, thus bypassing the cyclic dependency limitation, is that correct?

Good question and I believe it refers to the implementation in #12532. Unfortunately the answer is quite complicated. In that implementation, the variable set is special and allows you to inject properties from outside the geometry to create a variant. The implementation tries to make use as much as possible from links, subshapebinders, copy-on-change, and hiddenrefs to bypass dependency cycles to acquire what I call this application geometry interface (and it is pushing it).

I wouldn't state that a variable set has the ability to change properties, but properties in children are overridden because they have been rewritten with hiddenrefs to refer to the variable set that is linked to in App::Part. So, the App::Part has a property that is an external link to a variable set and the children refer to this property to access values from the property set.

In this project, I want to introduce an Exposed flag to properties to essentially acquire this level of parameterization anywhere. With #12532 it is only possible in App::Part but with an Exposed flag, it would also be possible to expose properties from for example a Part Design Body. This means that cyclic dependencies that variable sets currently overcome with hiddenrefs need to be solved at a more fundamental level, namely for exposed properties and this provides chances to improve on the copy-on-change behavior, the dangers of hiddenrefs, and hidden temporary files.

I hope it helps, please let me know if I need to explain more.

[yorikvanhavre]

I am sorry that I am insisting on the same things over and again Pieter, I hope you won't feel pissed, but I feel these things are important.

Unfortunately the answer is quite complicated

That's exactly where the problem lies, for me. I feel you are proposing a too complex solution for a problem that could be described and attacked in a much more self-contained manner. The FreeCAD core code has been complexified a lot already in the recent past, and this has not been really beneficial for FreeCAD. The core code design is (or at least was) very simple and clear, and this is a powerful thing of FreeCAD. The complexity goes into specific implementations (modules basically, but more generally anything that is contained to a specific domain).

The main "schema" of FreeCAD, how it works, its main structures and systems, should be kept very clear to all contributors, not only those with a master degree in programming. That's how we keep it a community-developed project and not a specialist niche. Also, the more the core becomes complex, the more problems it triggers down the road and the less controllable the whole thing is. You mention yourself the problems with hidden refs, for example. "Changing the execution model of FreeCAD" has a huge potential for problems and is certainly not something o be undertaken without a detailed plan and a strong and clear agreement among developers on how to do it.

Don't get me wrong, I agree fundamentally with the problems you are trying to solve, and it is totally valid for a grant. It would be desirable to have, for example, a model of a chair, where you could have a "Height" property, which would automatically update the heights properties of its components. But IMHO adding layers of complexity to the core code to achieve that is not desirable and not the right way to solve such a problem.

Again, this is just my personal opinion. I can totally be proved wrong.

[shaise]

Pieter,

In my opinion marking inside fields as "Exposed" is not a good idea. Aside from complexing the core it might miss the simplification of parameters. Here is my example, say we want a chair based on human size. you have an OUTSIDE parameter name human_heigth, the function goes

function F()
{
   height = human_height * 0.8
   width = human_height * 0.4
} 

if we use the expose method we need to expose width and height

function F(width, height)
{
}

Now for every variant we have to set width and height separately instead of changing only human_height. I'm personally against "injecting" values. It is always better if the object is referencing external parameter perhaps even as a part of formula Having an external "parameters" object is much less messing with core code since parameters today already can reference external parameters. I DO like the possibility to have a separate copy per linked part, so variants can be made as opposed to having a single parameter object like in most other cad SW. Having a separate parameters per part is good enough, we really do not need an option to make variants to each part::cylinder we create. Anything we want to make variation of, we can wrap in a part container. Any user who sees the "parameter" object under the part container KNOWS this is where he can enter and make variants.
I also suggest a "reset" function: when you press reset, all values from the parent of the linked part will be copied. So you can easily reset the variant.

I also think this option is much easier to implement and will give us a standardized way to have document parameters.

[pieterhijma]

I am sorry that I am insisting on the same things over and again Pieter, I hope you won't feel pissed, but I feel these things are important.

No, I don't feel pissed at all. You are right that these things are important and I completely understand your reservations.

Unfortunately the answer is quite complicated

That's exactly where the problem lies, for me. I feel you are proposing a too complex solution for a problem that could be described and attacked in a much more self-contained manner. The FreeCAD core code has been complexified a lot already in the recent past, and this has not been really beneficial for FreeCAD. The core code design is (or at least was) very simple and clear, and this is a powerful thing of FreeCAD. The complexity goes into specific implementations (modules basically, but more generally anything that is contained to a specific domain).

I'm certain that this needs support from core. A variant would be a form of a link and it is not for nothing that links, hiddenrefs, copy-on-change, etc. are implemented in core. The above functionality requires support from the core and that is what the core is for. I'm totally fine with moving anything that can be moved outside App to Mod/VarSets or Mod/Variants, but to generalize it means that it becomes a cross-cutting feature (which is good). The logical place for supporting this functionality is core.

The main "schema" of FreeCAD, how it works, its main structures and systems, should be kept very clear to all contributors, not only those with a master degree in programming. That's how we keep it a community-developed project and not a specialist niche. Also, the more the core becomes complex, the more problems it triggers down the road and the less controllable the whole thing is. You mention yourself the problems with hidden refs, for example. "Changing the execution model of FreeCAD" has a huge potential for problems and is certainly not something o be undertaken without a detailed plan and a strong and clear agreement among developers on how to do it.

Well, the core is already very difficult to understand, also for me. In my opinion the complexity of the core stems from the fact that the execution model has limitations. This proposal would provide a great opportunity to investigate how to support functionality such as variants and configurations better. This has the potential to have a more fundamental solution for this kind of functionality with the ultimate effect that the core becomes easier to understand and the more complex existing solutions are only used for backward compatibility. I think this path is better than leaving the core alone as much as possible because there are parts that only few understand. I've submitted #4 as well to improve everyone's understanding of the core, so I hope it is clear I'm committed to the goal of accessibility of the source to anyone. In fact, in the text above I've already stated in "Risks and Mitigation" that at the very least we gain more understanding of the problems at hand.

The resources in terms of programmers who fully understand potential solutions and the implications are scarce. I hope the committee recognizes that they have the opportunity to exchange the resource "funding" into one more "programmer" resource with a degree in computer science that understands potential solutions and the implications for this kind of functionality.

Don't get me wrong, I agree fundamentally with the problems you are trying to solve, and it is totally valid for a grant. It would be desirable to have, for example, a model of a chair, where you could have a "Height" property, which would automatically update the heights properties of its components. But IMHO adding layers of complexity to the core code to achieve that is not desirable and not the right way to solve such a problem.

As I said above, I think it has much potential to be a less complex solution than what is in core right now. And again, if there is anything that I can move to a separate module, I completely agree that this would be desirable.

Again, this is just my personal opinion. I can totally be proved wrong.

I've marked this myself as having quite some risk and even if the result is not ready to be merged, for example because it turns out that the core has to become even more complex than it is now, I'm certain that the community will learn from this work. In addition, it has potential to gain a better understanding of other related areas such as cyclic dependencies, granularity of dependency checking, and links.

[pieterhijma]

Dear Shai,

I think this goes a bit out of scope now because it is not about the proposal anymore but about the implementation. But it is good to exchange thoughts on this.

In my opinion marking inside fields as "Exposed" is not a good idea. Aside from complexing the core it might miss the simplification of parameters. Here is my example, say we want a chair based on human size. you have an OUTSIDE parameter name human_heigth, the function goes

function F()
{
   height = human_height * 0.8
   width = human_height * 0.4
} 

if we use the expose method we need to expose width and height

function F(width, height)
{
}

I think I understand what you are saying and I will react to that but let me first address some things: if the chair is based on the human height and that may be something you want to change, I think this should be part of the application geometry interface, so that value should be exposed and width and height are computed from that. This is very well possible. Since each property must have a value, there is always a default value, so it would become more like this:

function F(human_height=175) {
}

You don't have to set the value, but if you want to create a non-standard variant, human_height is what you should change.

Having an outside parameter human_height breaks modularity. I show that in the video in this comment at 2:04. By the way, having parameters inside don't help in having to specify things only once as this video also illustrates. So, if you exchange the document object chair with anyone, the document object that stores human_height has to be exchanged as well. Since these values are typically stored in global document objects (in the case of the video in a spreadsheet), the user will get irrelevant information (see 2:36 - 3:34 in the video).

Now for every variant we have to set width and height separately instead of changing only human_height. I'm personally against "injecting" values. It is always better if the object is referencing external parameter perhaps even as a part of formula Having an external "parameters" object is much less messing with core code since parameters today already can reference external parameters. I DO like the possibility to have a separate copy per linked part, so variants can be made as opposed to having a single parameter object like in most other cad SW. Having a separate parameters per part is good enough, we really do not need an option to make variants to each part::cylinder we create. Anything we want to make variation of, we can wrap in a part container. Any user who sees the "parameter" object under the part container KNOWS this is where he can enter and make variants. I also suggest a "reset" function: when you press reset, all values from the parent of the linked part will be copied. So you can easily reset the variant.

I try to show in the video that it is not a very satisfying solution to rely on external parameters because it breaks modularity. Typically, my FreeCAD files have a single master file called Params.FCStd with a Params spreadsheet that drives all the parts in my assembly. Having it in a separate file solves cyclic dependencies, but all files ultimately depend on this spreadsheet. This results in a situation where the separate parts in other files are not modular anymore and rely on the parameter file. Often the part uses only a subset of all parameters specified, so most is irrelevant for the part.

Only limiting this to App::Part was one of the first criticisms I got with the variable set implementation. I think it makes sense to expose parameters on each level. For example, suppose you have a part consisting of body A and body B. Both A and B can be parameterized independently. Suppose you need a part with an A with length 10 and a B with length 20. And you need a part with an A of length 20 and a B of length 40. This is very easy to do if A and B expose their lengths to be set inside the part. Otherwise, you would need to copy A and B in each part and modify lengths inside the design.

I hope both the video mentioned earlier and its part 2 can convince you to at least investigate this option as it has many benefits.

I also think this option is much easier to implement and will give us a standardized way to have document parameters.

I'm not so sure it is easier to implement. It would require copying the parameters and somehow making sure that the linked geometry uses that. This would be a challenge and also needs support from core. And if the execution model doesn't change, it will rely on the same mechanisms that already exist. So, in my humble opinion this solution has drawbacks, is less powerful, and it would not form a fundamental solution that can potentially help us reduce complexity in core.

[shaise]

Hi @pieterhijma ,

Thank you for the video, it explains a lot. First of all I now understand the the 'exposed' variables are ONLY from the variable-set and not from inside parts (I initially thought you plan to 'expose' say the 'length' field inside a Pad function for example) In this case of course please forgive me about the example with human_height, it is not relevant at all. The variable set object is exactly as I imagined the "parametes" object. The only difference is the usage of variable sets. In your implementation for each variant you need a separate variable set. In my thought whenever you make a link to another part, you will automatically create a new variable set inside the link, with only the exposed parameters enabled. This way no need to create a new set manually, then manually link them to the part.

Of course, I can see cases where your implementation have benefits over my idea. Regarding complexity, I really can not say much as I'm not that familiar with the core, But I guess @yorikvanhavre does, and maybe you can give us a general notion what parts/classes in the core will be affected.

Sorry for misunderstanding shai

[pieterhijma]

Hi @pieterhijma ,

Thank you for the video, it explains a lot. First of all I now understand the the 'exposed' variables are ONLY from the variable-set and not from inside parts (I initially thought you plan to 'expose' say the 'length' field inside a Pad function for example) In this case of course please forgive me about the example with human_height, it is not relevant at all. The variable set object is exactly as I imagined the "parametes" object. The only difference is the usage of variable sets. In your implementation for each variant you need a separate variable set. In my thought whenever you make a link to another part, you will automatically create a new variable set inside the link, with only the exposed parameters enabled. This way no need to create a new set manually, then manually link them to the part.

It is true that you need a separate variable set for each variant but there are some thoughts on alleviating this, for example @Nomanor suggested to allow subsets in this comment. Additionally, as also happened in the second video, you only need a varset for each unique variant. If you reuse a variant, you can simply use a link. Thirdly, the values that drive a variant have to come from somewhere. And finally, I think it would be good to have a powerful tool to manage properties in the future. I've already made an initial proposal for such a tool in #13112. The idea for the future is that you can "refactor" properties and variable sets easily for creating variants and other things. I hope that with the collection of these features we can move away from spreadsheets and make use of the powerful property system that FreeCAD already has for this level of parameterization.

Of course, I can see cases where your implementation have benefits over my idea. Regarding complexity, I really can not say much as I'm not that familiar with the core, But I guess @yorikvanhavre does, and maybe you can give us a general notion what parts/classes in the core will be affected.

Great, I'm glad you can recognize the benefits. I would estimate now that in core it would entail changes in DocumentObject.cpp for tweaks to the execution model. In Property.cpp we would need to add the Exposed flag. The dependency checking is in Document.cpp and DocumentObject.cpp (inlists, outlists) and a tiny bit in PropertyExpressionEngine.cpp. ObjectIdentifier.cpp may allow me to redirect property access of exposed properties to other properties and in Link.cpp we may need some support for a special variant link. However, if in any way possible, I would agree with Yorik that if we can move VarSet.cpp, and the logic for variants to Mod/Variant or something similar, then this clearly good to do.

Sorry for misunderstanding shai

No problem at all. I've had this problem in mind for more than a year now and I had to condense this proposal in perhaps 500 words, so I can very well imagine that it is difficult to judge. I was triggered by Ondsel's blog post who decided to hire me to work on this after I explained my ideas on this topic. See some of my earlier comments here. The essence is still the same except for the realization that it couldn't work on the file level because links work on the document object level. After a first implementation in #12532 it is clear that FreeCAD's core can support this functionality in only a limited way and it would be better if there is more fundamental support that doesn't rely on copying, hiddenrefs and temporary hidden files.

[shaise]

Hi @pieterhijma ,

I hope that with the collection of these features we can move away from spreadsheets and make use of the powerful property system that FreeCAD already has for this level of parameterization.

Yes indeed! That's why I think this feature is important.

some more clarifications if you may:

1. I see the variants work on shape binders. I guess it works also on Part Links? what else?
2. When a varset is exposed, a variant can point to another varset. what happens if this other varset have different parameter names?
3. Why do we need the "Expose" at all? Why cant it be exposed by default? Its not like the parts are 'black-boxes' If someone shared a part with me, I can always set the expose to true. This way we do not have to add this expose feature in the core. Now whenever I link a part I can leave it as is if I don't want to change it, or switch the varset to get a variant.

Thank you! shai

[pieterhijma]

Hi @pieterhijma ,

I hope that with the collection of these features we can move away from spreadsheets and make use of the powerful property system that FreeCAD already has for this level of parameterization.

Yes indeed! That's why I think this feature is important.

Great.

some more clarifications if you may:

1. I see the variants work on shape binders. I guess it works also on Part Links? what else?

You're talking about the current implementation. This works with subshapebinders and links. For the improved implementation, I would hope there is going to be a Part::Variant that assumes the variant role and makes subshapebinders and copy-on-change links obsolete. Another possibility I can see is that App::Link supports variants not by means of copy-on-change but with the Exposed properties.

2. When a varset is exposed, a variant can point to another varset. what happens if this other varset have different parameter names?

This would not be an "equivalent varset". You would get an error message.

3. Why do we need the "Expose" at all? Why cant it be exposed by default? Its not like the parts are 'black-boxes' If someone shared a part with me, I can always set the expose to true. This way we do not have to add this expose feature in the core. Now whenever I link a part I can leave it as is if I don't want to change it, or switch the varset to get a variant.

Very good question. In a design there are often more values that you want to name. For instance, in your example you had the values 0.4, 0.8, human_height, width, and height. I would always name values such as 0.4, for instance factor_width, or ratio_width. However, when exchanging the design, the designer may want to express intent. The designer can mark human_height as exposed, therefore communicating to the user of the design that the user can (or should) parameterize the design by means of adjusting this human_height parameter. This is what I mean that exposing properties creates a contract between designer and user.

The concept of contracts is common in programming which basically boils down that the function defines how it should be called by means of invariant, pre-conditions, post-conditions, sometimes even formally encoded. For example, here is an old proposal for contracts in Python. In most programming languages and programs, the contracts are informally described in docstrings, for example: "This function should not be called with an integer > 0"

In the example above, you assume that the geometry you are using is on your filesystem and so you can change everything, but I can very well imagine a future where the geometry you are using comes from the internet on the fly.

In any case, FreeCAD files and parts have no standardized way (or one at all if you ask me) to communicate to the user of a part how to parameterize the part. Some use spreadsheets, Dynamic Data, Path property bags for parameterization, spreadsheets configurations or Dynamic Data configurations, properties in objects, named constraints in Sketcher and what have you. I think it makes sense to have a feature supported all across FreeCAD to be able to communicate the important part of design intent, namely how to parameterize an object.

What you are describing above sounds similar to the Assembly 4 variants that are based on temporary hidden documents. However, that implementation has its drawbacks and I can very well imagine that to circumvent those drawbacks, you would need to make changes in the core anyway.

However, I'm completely open to the idea that if we find a way for an implementation that hardly touches the core, then this would be favorite. I think the chances for that are low, but perhaps I can find a way.

[shaise]

Thanks!!

[brittanyb]

Is it viable to consider an inheritance system for properties, and a computation UUID for this job? I lack knowledge about the core functionality, so might make some assumptions, but thought it was worth chiming in when thinking about this suggestion.

As an example, I'll give a use case of how we might create variant parts, considerate of a developer-centric approach.

For the base variant object:

• We create a 'Base variant' object which is just an App::Part that contains an App::PropertyContainer object, and a PartDesign::Body object.
• The App::Part base data for a 'Base variant' object includes a new App::PropertyInheritedBodyDAG property, which is a direct pointer to the design DAG of the PartDesign::Body, and the property has an associated Computation UUID. -The App::Part also has a new App::PropertyInheritedPropertyContainer property, which can behave as a link to a property container, residing in a different object. We set this link to the App::PropertyContainer object. This avoids the cyclic reference issue by allowing the configuration properties to exist within the child App::PropertyContainer, but having an access point in the parent App::Part. The App::PropertyInheritedPropertyContainer also has its own associated Computation UUID. This new property creates a copy of the contents of the property container into the App::Part base properties.
• All properties and configuration data, including their lists/selection values, are added to the App::PropertyContainer, and utilised within the PartDesign::Body.

For the variant object:

• We create a standard App::Part object, that has a new App::PropertyInheritedPropertyContainerMutant, which links to the 'Base variant' objects 'Properties' App::PropertyInheritedPropertyContainer, and has its own associated Computation UUID and contains 'initial value' data for each property to test whether it should be mutated during updates.
• The App::Part object contains a PartDesign::Body that has a base property PartDesign::PropertyInheritedBodyDAGMutant, which links to the 'Base variant' objects App::PropertyInheritedBodyDAG property, and has its own associated Computation UUID. The user shouldn't be able to 'edit' any of the 'Base variant' DAG features, but should be able to delete them, and to be able to insert their own new features into the DAG. We can also store data regarding these feature deletions/additions, so that they can be re-inserted during updates.

The idea here is:

• When the 'Base variant' property container receives an update, the new data is copied within the App::Part property App::PropertyInheritedPropertyContainer, and its Computation UUID is updated.
• When a variant object is re-computed, App::PropertyInheritedPropertyContainerMutant property's Computation UUID is compared to that of the App::PropertyInheritedPropertyContainer property's Computation UUID. If they differ, then a copy will occur from the latter to the former, and its Computation UUID updated.
• During this re-compute/copy cycle, we will also check the 'current' property data to the 'initial' property data. This will allow the user to update parameters at will and preserve them across copy updates. For this reason a 'reset' option should be present on each property.
• The same re-compute/copy cycle occurs for the App::PropertyInheritedBodyDAG and App::PropertyInheritedBodyDAGMutant. Effectively, this will cause a copy of the App::PropertyInheritedBodyDAG data, directly into the PartDesign::Body of the 'Variant' when the 'Base variant' object changes, while preserving the feature deletions/additions the user made within this 'Variant'.

This still doesn't address UI, and would probably need:

• A button for the 'Base variant' and 'Variant' parts to be added.
• An App::VariantContainer object, that would contain these App::Part objects.
• An editor for the App::VariantContainer, similar to the Material container system. This'd allow users to add their 'Base variant' part, add any 'Variant' parts, and update their respective properties, while managing the link chains I've suggested above automatically.

This method does use copies and links, but constrains the work needed to existing caching and property value mechanisms already in FreeCAD. I think it sits pretty close to the existing paradigms we have, but avoids temporary files, cyclic references, and isn't causing performance loss unless it needs to happen anyway, due to the Computation UUID. Since this method relies on standard App::Part objects, they could also be used in the upcoming Assembly workbench without further integration, using links to the existing parts.

[pieterhijma]

Dear @brittanyb,

Thank you for the detailed suggestions! Sorry for the late reaction, I needed some time to wrap my head around this idea. It is certainly possible to consider this direction, but it is quite a different direction. I'm not sure it would be an improvement to the current proposal, though, because of the following reasons:

• It does not generalize the usage of "exposing properties", but it ties it to an App::Part and an PartDesign::Body (but I would hope this could be generalized a bit).
• It does not address the main criticism of the current plan that it requires core changes. Your proposed implementation might even need more core changes than my proposal albeit in different places.
• It follows the familiar pattern of working around FreeCAD's limitations to acquire powerful features, see below for an explanation.

I'll react in-line as well:

Is it viable to consider an inheritance system for properties, and a computation UUID for this job? I lack knowledge about the core functionality, so might make some assumptions, but thought it was worth chiming in when thinking about this suggestion.

As an example, I'll give a use case of how we might create variant parts, considerate of a developer-centric approach.

For the base variant object:

• We create a 'Base variant' object which is just an App::Part that contains an App::PropertyContainer object, and a PartDesign::Body object.

• The App::Part base data for a 'Base variant' object includes a new App::PropertyInheritedBodyDAG property, which is a direct pointer to the design DAG of the PartDesign::Body, and the property has an associated Computation UUID. -The App::Part also has a new App::PropertyInheritedPropertyContainer property, which can behave as a link to a property container, residing in a different object. We set this link to the App::PropertyContainer object. This avoids the cyclic reference issue by allowing the configuration properties to exist within the child App::PropertyContainer, but having an access point in the parent App::Part. The App::PropertyInheritedPropertyContainer also has its own associated Computation UUID. This new property creates a copy of the contents of the property container into the App::Part base properties.

This way of circumventing cyclic dependencies is smart, I think. I'm a bit confused and can currently understand this in two ways: The property with type App::PopertyInheritedPropertyContainer is responsible for keeping the properties of the App::Part base properties up-to-date with changes to the property container it links to. So, in this scenario, the App::PropertyInheritedPropertyContainer points to a property container outside the App::Part. This would be quite a special property then and would definitely need support in core. The other scenario is that the property always remains pointing to the property container inside the App::Part just to give an entry point to change properties from outside the part. In this scenario I don't understand your statement about "receiving an update" below, however.

The copy of the contents of the property container into the App::Part does not seem not to be used in the rest of the idea, except for what I came up myself as an answer to how to store initial values. Do I understand that correctly?

• All properties and configuration data, including their lists/selection values, are added to the App::PropertyContainer, and utilised within the PartDesign::Body.

A drawback is that you can only make a single property container with configuration data. This is similar to Assembly 4 that has the usage of one Variables container hardcoded. Especially for more complex situation where you have sub-assemblies, this would not be ideal.

For the variant object:

• We create a standard App::Part object, that has a new App::PropertyInheritedPropertyContainerMutant, which links to the 'Base variant' objects 'Properties' App::PropertyInheritedPropertyContainer, and has its own associated Computation UUID and contains 'initial value' data for each property to test whether it should be mutated during updates.

I think I understand, but I'm not sure. I think what you mean is that the initial value data is stored inside the App::Part objects as base properties (as I mentioned above)? Otherwise, it would be difficult to capture a set of properties inside a single property.

• The App::Part object contains a PartDesign::Body that has a base property PartDesign::PropertyInheritedBodyDAGMutant, which links to the 'Base variant' objects App::PropertyInheritedBodyDAG property, and has its own associated Computation UUID. The user shouldn't be able to 'edit' any of the 'Base variant' DAG features, but should be able to delete them, and to be able to insert their own new features into the DAG. We can also store data regarding these feature deletions/additions, so that they can be re-inserted during updates.

This sounds very complicated to achieve. I would assume that the body is then some sort of initial copy of the Body of the base variant. In that sense, I understand why you shouldn't be able to edit the DAG features. However, I don't understand how you would be able to insert new features into the DAG. So, I guess you change the body inside the variant and therefore create a variant? Storing data regarding these changes would be very difficult as well: where do we store that administration in such a way that it is serialized as well. This would be a major change in core as well, I think.

The idea here is:

• When the 'Base variant' property container receives an update, the new data is copied within the App::Part property App::PropertyInheritedPropertyContainer, and its Computation UUID is updated.

I don't understand completely. The App::PropertyInheritedPropertyContainer is linked to the 'Base variant' property container, right? Additionally, "receiving an update" is not completely clear to me. You can set a value, but this seems to assume we can distinct setting a value directly into the property container inside the part or through the link. I don't think that is possible right now and would require changes in core.

• When a variant object is re-computed, App::PropertyInheritedPropertyContainerMutant property's Computation UUID is compared to that of the App::PropertyInheritedPropertyContainer property's Computation UUID. If they differ, then a copy will occur from the latter to the former, and its Computation UUID updated.

Yes, I understand. So, the variant is based on copies which is a drawback compared to my proposal.

• During this re-compute/copy cycle, we will also check the 'current' property data to the 'initial' property data. This will allow the user to update parameters at will and preserve them across copy updates. For this reason a 'reset' option should be present on each property.

Yes, got it. A 'reset' option would be a feature that is currently not present. It would be difficult to capture that outside of the core of FreeCAD I think.

• The same re-compute/copy cycle occurs for the App::PropertyInheritedBodyDAG and App::PropertyInheritedBodyDAGMutant. Effectively, this will cause a copy of the App::PropertyInheritedBodyDAG data, directly into the PartDesign::Body of the 'Variant' when the 'Base variant' object changes, while preserving the feature deletions/additions the user made within this 'Variant'.

The first part I understand, but the second seems problematic to me. Again, I think this would require quite some administration and it depends on names and features being stable. This is going to be improved with toponaming, but it might be dangerous to rely on it.

This still doesn't address UI, and would probably need:

• A button for the 'Base variant' and 'Variant' parts to be added.

• An App::VariantContainer object, that would contain these App::Part objects.

• An editor for the App::VariantContainer, similar to the Material container system. This'd allow users to add their 'Base variant' part, add any 'Variant' parts, and update their respective properties, while managing the link chains I've suggested above automatically.

Yes, so in my humble opinion also quite some changes.

This method does use copies and links, but constrains the work needed to existing caching and property value mechanisms already in FreeCAD. I think it sits pretty close to the existing paradigms we have, but avoids temporary files, cyclic references, and isn't causing performance loss unless it needs to happen anyway, due to the Computation UUID. Since this method relies on standard App::Part objects, they could also be used in the upcoming Assembly workbench without further integration, using links to the existing parts.

Yes, I see what you mean and I think it circumvents cyclic dependencies and temporary hidden files, but I do think the design has some problems that I don't see how to solve immediately, especially regarding administration. I think the idea is actually quite similar to what already exists in FreeCAD with copy-on-change links, if you set it to "Tracking". This also maintains an elaborate administration of changes to the original version with a CopyOnChangeGroup and UUIDs. The only difference is that the tracking copy-on-change links don't provide a way to set properties from the outside except for hidden references. This proposal would have a similar drawback to tracking copy-on-change: each time you do an update, the variant geometry will be called differenty because a copy was created. This makes the variant not really a first-class citizen because it cannot easily be referenced to from outside because of changing names.

Thank you very much for the elaborate suggestions. The ideas alone are already very useful. For this specific implementation idea, though, I think it follows the same pattern that the whole link system followed as well: Try to make powerful features work making use as much as possible from the current infrastructure, introducing features such as hiddenrefs or elaborate administration schemes to work around the limitations of the current execution scheme. This is essentially what I tried to do in #12532 as well, running into the limitations. I would be more in favor of looking for a more fundamental solution along the lines that is proposed here that allows us to acquire those powerful features without working around the limitations, but by making FreeCAD inherently capable of this kind of functionality.

I'm curious to hear your thoughts.

[brittanyb]

Thanks Pieter, I appreciate you giving me your time and energy. For some context behind my reply - my comments come from having not done FreeCAD development before. I'm a relatively new user (probably about 2 months experience) that recently experimented with variant parts, and settled on my current workflow, which uses shape binders, base features, hiddenref's, and driving properties in parent parts from the children objects with expressions. I'm happy with the majority of this, albeit with some serious need for UI to improve the flow, and the desire to remove hiddenref's and temporary files.

My work lead me to these feelings:

• Why do we even need hiddenref functionality? If we were to treat properties as their own individual entity on a DAG, then it would be ideal enough to have properties depend on one another, without the parent object being concerned with its properties. The evaluation of cyclic dependency would then only need to be concerned as to whether a property update will trigger a loop of updates, as opposed to making a broad assumption that two objects having any back and forth dependency between their children is an issue that needs to be stopped. I imagine there might be something I'm missing about FreeCAD's core, though, as to why this might be a harder issue to manage than I'm making it out.
• Property grouping, management and updates being a well supported design paradigm with more support in FreeCAD's UI would be one of the most powerful additions to FreeCAD I would fall in love with immediately.
• I wish we didn't have to rely on links. They do feel brittle between files. This is why the 'copy and mutate' paradigm works so well, in my opinion. When I think about an example such as a PartDesign::Body and its constituent features to yield a solid part, I don't immediately see a method for allowing inherited properties without either one of two options: A deep copy, or, a pointer. The deep copy seems to yield the most flexibility, and risk reduction, when supported by a well-designed system, and the pointer seems ideal in a given file, and very non-ideal if outside of that file. The only idea I can come up with for a well-encapsulated pointer system is to separate it entirely from the files themselves, and obfuscate them from the user by hiding them in their application data, which is a terrifying thought.

On to replies:

This way of circumventing cyclic dependencies is smart, I think. I'm a bit confused and can currently understand this in two ways: The property with type App::PopertyInheritedPropertyContainer is responsible for keeping the properties of the App::Part base properties up-to-date with changes to the property container it links to. So, in this scenario, the App::PropertyInheritedPropertyContainer points to a property container outside the App::Part. This would be quite a special property then and would definitely need support in core. The other scenario is that the property always remains pointing to the property container inside the App::Part just to give an entry point to change properties from outside the part. In this scenario I don't understand your statement about "receiving an update" below, however.

'Receiving an update' was my way of saying that the user updates the property container, either by adding, removing or editing a property.

I think the way to think about it is that I'm suggesting a top-down approach to dependency with 'inheritance' whereas I feel your 'expose' method is more bottom-up. For 'expose', a signal is emitted, and any attached listener would respond appropriately. For 'inheritance', it's the idea that we avoid trying to do all the updates at once to any listeners, and instead, allow the property to change, and only be read by any listeners when those listeners themselves are re-computed. This gets around the 'large re-compute' issue, without being much of a change to your core proposal.

The copy of the contents of the property container into the App::Part does not seem not to be used in the rest of the idea, except for what I came up myself as an answer to how to store initial values. Do I understand that correctly?

Partially, yes. In this case, I'm suggesting a simple 'pointer' system for exchange of data within a file, as a file is inherently self-contained. I'm sure it'd have other uses, but here, it's just used for organisation. The idea being that the properties are 'exposed' to a parent, simply by the parent having an expression that directly reflects the contents of a child. I think I may've mis-used the word 'copy' here, as I'd meant to suggest it was only a direct 'pointer', apologies.

A drawback is that you can only make a single property container with configuration data. This is similar to Assembly 4 that has the usage of one Variables container hardcoded. Especially for more complex situation where you have sub-assemblies, this would not be ideal.

When I was thinking about my idea last night, I had similar thoughts. I'm suggesting a very specific use-case to achieve variants, whereas your idea ties more into a generalized system. I think you're more on track with the best answer in that regard.

I think I understand, but I'm not sure. I think what you mean is that the initial value data is stored inside the App::Part objects as base properties (as I mentioned above)? Otherwise, it would be difficult to capture a set of properties inside a single property.

I'm suggesting two values to track. The current value, which is whatever happens to be the inherited properties mutated value at the time, and the initial value, which is an inherited property's way of capturing what the initial state of the property's value was at the time of the initialized link between the inherited property and the underlying property.

The only question is whether the initial value should be stored in the property, or inherited property, and I believe the inherited property is the best place for it to live. This is because transferring this data point each time would cause extra re-compute, whereas reset would be the way to manually re-compute the initial value if the original property has changed. I also leaned in this direction because I can easily see the inherited property being used in a different file to the original property, and I prefer to use copying to avoid link breakage causing the new inherited property to be unable to be re-computed. In the case of link breakage with this option, the inherited property continues to function properly, but simply stops having its initial value property updated.

The first part I understand, but the second seems problematic to me. Again, I think this would require quite some administration and it depends on names and features being stable. This is going to be improved with toponaming, but it might be dangerous to rely on it.

I don't believe so. By capturing the structure of the DAG that builds the solid, as opposed to the geometry of the solid, we're not relying on the solid geometry, nor any specific feature details. This is also why I believe in using copy when the user updates the base variant PartDesign::Body. If we imagine the base variant solid as a square sketch, padded, and then another square sketch, which is then pocketed, then we can think of two different areas of concern. If the base variant's pad's square sketch has its length parameter updated, then it runs the risk of toponaming issues in anything that points directly to the pad feature. If, however, we're to copy that sketch and feature on change, into the variant part, then we circumvent this issue. That brings the secondary issue, which is, for example, in the variant part, we decide to insert a pad between its base variant inherited pad/pocket features. In this case, when the pad/pocket operations change in the base variant, are we then able to reliably copy that information into the variant part, and re-insert the pad to the correct position? I believe the answer is yes, provided that this step follows best practices to avoid toponaming problems.

--

I think that there's a mix of ideas here that looks to be converging on an ideal answer. For one question I have, I may have not properly understood this part of your proposal - Is your suggestion that to overcome the issue of copying and linking that you will research an answer, or have I missed the context of this as a resolved issue? I'm thinking as a programmer, and I see this as a fundamental issue of having a variable we either update permanently, or have the variable be a pointer to another variable. I see the most flexible answer as copy and track, considering that it allows the pointer-style updates, but allows the pointer to break as the copied variable will always remain.

For a little further thought - I think part of the issue is how FreeCAD lacks some core structure to how the file interoperability is managed. There's very little description of what a file is meant to be, and as a result, an issue in determining what the best way to structure a project or part should be. Some general practice has a part per file, except in the case of assemblies, where part links are grouped into a single .fcstd file. If we decide to describe variant parts/assemblies as their own distinct file standard of file structure, maybe even with their distinctive name, .fcprt and .fcasm, then it acts as a way to separate the concerns of management of these complicated systems from such a generic standard of file interaction.

In that case, I could see .fcprt making good use of an exposed property system and for any .fcasm to be responsible for maintaining a link to the .fcprt. Still, I don't see a good way to get away from copying, here. If we had an extrusion.fcprt and a extrusion_frame.fcasm, then the ideal would be to expose the 'length' property of the .fcprt, and to have it set within the .fcasm. This puts the burden on the assembly to maintain the variant, as opposed to the variant part file. For the amount of different ways to define that length, it would be cumbersome to have those definitions/their linkage tracked within the file. But still, we would need the .fcasm to keep a local object copy for each extrusion body of each length definition.

The best idea I can come up with for the copy flow here would be that the assembly file on any re-compute first checks whether the link can be found to the variant part. If it's not there, we won't pull any updates, and will just rely on our local copy for the rest of the re-compute. If the link is available, and the file is open (or perhaps we create a way to pull relevant file information from the part file without having to have the file open [oh no, temporary files..]), then we check that object's compute UUID to check whether or not we need to copy anything in from the variant part to the assembly file.

I also think if we have a .fcasm and .fcprt file communications standard, then it will make it a lot easier to pull these properties without having to keep every single file open all of the time, and easier manage the properties. An example: a single panel.fcprt stands on four standoff.fcprt in each corner, arranged in the supported_panel.fcasm file. We conclude that the standoff.fcprt 'length' parameter should be driven by the panel.fcprt length parameter. This is a genuine use-case, as we could easily conclude that we'd want the corner mounts of our panel to be larger as the length of the panel increases. In this case, perhaps we need to conclude that we should be able to have each 'exposed' property in the .fcprt act as having both driving/driven behaviour. This way, we would be able to allow the panel.fcprt to describe its 'minimum_length' property to '250mm', create a 'panel_length_addon' property in supported_panel.fcasm set to '150mm', and use that to drive the panel.fcprt 'length' property as the 'minimum_length + panel_length_addon'. This works quite well, as the variant part suggests it requires some input from the inheriting assembly. We can then pass that 'length' back to the parent, and have it drive the standoff.fcprt as having a 'minimum_length' of '10mm', and a 'length' of 'minimum_length * panel_length / 10'.

Again, thank you for your time. I think you've given this a lot of thought and that you're definitely the right person to handle this system change, I do wonder if there'll be some unavoidable bumps though. I'd also be interested in helping with the UI for the changes (though I'm not looking for money).

[pieterhijma]

Thanks Pieter, I appreciate you giving me your time and energy. For some context behind my reply - my comments come from having not done FreeCAD development before. I'm a relatively new user (probably about 2 months experience) that recently experimented with variant parts, and settled on my current workflow, which uses shape binders, base features, hiddenref's, and driving properties in parent parts from the children objects with expressions. I'm happy with the majority of this, albeit with some serious need for UI to improve the flow, and the desire to remove hiddenref's and temporary files.

Right, those are advanced features but can get you to something like a variant.

My work lead me to these feelings:

• Why do we even need hiddenref functionality? If we were to treat properties as their own individual entity on a DAG, then it would be ideal enough to have properties depend on one another, without the parent object being concerned with its properties. The evaluation of cyclic dependency would then only need to be concerned as to whether a property update will trigger a loop of updates, as opposed to making a broad assumption that two objects having any back and forth dependency between their children is an issue that needs to be stopped. I imagine there might be something I'm missing about FreeCAD's core, though, as to why this might be a harder issue to manage than I'm making it out.

Good question to which I don't know the answer but I can make a couple of guesses:

• A DAG is something that simply disallows cycles, so you can't get into infinite loops when recomputing, so historically this might have been the most logical solution, not anticipating that it may at some point in the future be a limitation for specific use-cases.
• Properties are seen as to drive changes in geometry in the document objects. Since the new shapes have to be computed in the document objects, that was deemed the right granularity for dependencies.
• It may have to do with spreadsheets that can contain values but are not strictly represented by properties, so the easy way out is to do recompute based on objects.

I agree. It might be worth changing the granularity of dependencies to properties and relaxing the problem of cyclic dependencies. If we could do a fixed-point computation, that might be a solution, but I'm not so sure how we can define stability for properties if they allow ranges of values. This is definitely something that can be investigated.

• Property grouping, management and updates being a well supported design paradigm with more support in FreeCAD's UI would be one of the most powerful additions to FreeCAD I would fall in love with immediately.

I made a couple of steps there: #13112 and #13038.

• I wish we didn't have to rely on links. They do feel brittle between files. This is why the 'copy and mutate' paradigm works so well, in my opinion. When I think about an example such as a PartDesign::Body and its constituent features to yield a solid part, I don't immediately see a method for allowing inherited properties without either one of two options: A deep copy, or, a pointer. The deep copy seems to yield the most flexibility, and risk reduction, when supported by a well-designed system, and the pointer seems ideal in a given file, and very non-ideal if outside of that file. The only idea I can come up with for a well-encapsulated pointer system is to separate it entirely from the files themselves, and obfuscate them from the user by hiding them in their application data, which is a terrifying thought.

Hm, I may not understand. I think most really like links, including me. They are very flexible and save space in complex assemblies. I think they are already pretty much separated from the files because they work on the level of document objects. I agree they are brittle when dealing with broken links, but that is a logical consequence of being links. Copy-and-mutate seems to be inefficient for many variants with small changes.

On to replies:

This way of circumventing cyclic dependencies is smart, I think. I'm a bit confused and can currently understand this in two ways: The property with type App::PopertyInheritedPropertyContainer is responsible for keeping the properties of the App::Part base properties up-to-date with changes to the property container it links to. So, in this scenario, the App::PropertyInheritedPropertyContainer points to a property container outside the App::Part. This would be quite a special property then and would definitely need support in core. The other scenario is that the property always remains pointing to the property container inside the App::Part just to give an entry point to change properties from outside the part. In this scenario I don't understand your statement about "receiving an update" below, however.

'Receiving an update' was my way of saying that the user updates the property container, either by adding, removing or editing a property.

I think the way to think about it is that I'm suggesting a top-down approach to dependency with 'inheritance' whereas I feel your 'expose' method is more bottom-up. For 'expose', a signal is emitted, and any attached listener would respond appropriately. For 'inheritance', it's the idea that we avoid trying to do all the updates at once to any listeners, and instead, allow the property to change, and only be read by any listeners when those listeners themselves are re-computed. This gets around the 'large re-compute' issue, without being much of a change to your core proposal.

Right, yes, I see what you mean with the difference in top-down and bottom-up, but I think in the end it doesn't matter with regard to recompute. What has to be recomputed has to be recomputed in the end.

The copy of the contents of the property container into the App::Part does not seem not to be used in the rest of the idea, except for what I came up myself as an answer to how to store initial values. Do I understand that correctly?

Partially, yes. In this case, I'm suggesting a simple 'pointer' system for exchange of data within a file, as a file is inherently self-contained. I'm sure it'd have other uses, but here, it's just used for organisation. The idea being that the properties are 'exposed' to a parent, simply by the parent having an expression that directly reflects the contents of a child. I think I may've mis-used the word 'copy' here, as I'd meant to suggest it was only a direct 'pointer', apologies.

Right, thanks.

A drawback is that you can only make a single property container with configuration data. This is similar to Assembly 4 that has the usage of one Variables container hardcoded. Especially for more complex situation where you have sub-assemblies, this would not be ideal.

When I was thinking about my idea last night, I had similar thoughts. I'm suggesting a very specific use-case to achieve variants, whereas your idea ties more into a generalized system. I think you're more on track with the best answer in that regard.

Right.

I think I understand, but I'm not sure. I think what you mean is that the initial value data is stored inside the App::Part objects as base properties (as I mentioned above)? Otherwise, it would be difficult to capture a set of properties inside a single property.

I'm suggesting two values to track. The current value, which is whatever happens to be the inherited properties mutated value at the time, and the initial value, which is an inherited property's way of capturing what the initial state of the property's value was at the time of the initialized link between the inherited property and the underlying property.

The only question is whether the initial value should be stored in the property, or inherited property, and I believe the inherited property is the best place for it to live. This is because transferring this data point each time would cause extra re-compute, whereas reset would be the way to manually re-compute the initial value if the original property has changed. I also leaned in this direction because I can easily see the inherited property being used in a different file to the original property, and I prefer to use copying to avoid link breakage causing the new inherited property to be unable to be re-computed. In the case of link breakage with this option, the inherited property continues to function properly, but simply stops having its initial value property updated.

Right, yes, I don't know how we could associate a set of properties to a link property though. I'm sure it's possible but FreeCAD may have little support for this currently.

The first part I understand, but the second seems problematic to me. Again, I think this would require quite some administration and it depends on names and features being stable. This is going to be improved with toponaming, but it might be dangerous to rely on it.

I don't believe so. By capturing the structure of the DAG that builds the solid, as opposed to the geometry of the solid, we're not relying on the solid geometry, nor any specific feature details. This is also why I believe in using copy when the user updates the base variant PartDesign::Body. If we imagine the base variant solid as a square sketch, padded, and then another square sketch, which is then pocketed, then we can think of two different areas of concern. If the base variant's pad's square sketch has its length parameter updated, then it runs the risk of toponaming issues in anything that points directly to the pad feature. If, however, we're to copy that sketch and feature on change, into the variant part, then we circumvent this issue. That brings the secondary issue, which is, for example, in the variant part, we decide to insert a pad between its base variant inherited pad/pocket features. In this case, when the pad/pocket operations change in the base variant, are we then able to reliably copy that information into the variant part, and re-insert the pad to the correct position? I believe the answer is yes, provided that this step follows best practices to avoid toponaming problems.

Ok, I see what you mean, but this makes both assumptions on FreeCAD and on the users, I think: It assumes that FreeCAD can recreate the structure of the body, capturing exactly those properties that happen to drive the design to recreate it. Currently at least that would be difficult to achieve I believe. Second, it assumes that users follow best practices to avoid toponaming problems. The fact that FreeCAD is investing so much time in toponaming, shows that users have difficulty to avoid these problems.

--

I think that there's a mix of ideas here that looks to be converging on an ideal answer. For one question I have, I may have not properly understood this part of your proposal - Is your suggestion that to overcome the issue of copying and linking that you will research an answer, or have I missed the context of this as a resolved issue? I'm thinking as a programmer, and I see this as a fundamental issue of having a variable we either update permanently, or have the variable be a pointer to another variable. I see the most flexible answer as copy and track, considering that it allows the pointer-style updates, but allows the pointer to break as the copied variable will always remain.

Yes, this proposal should be seen as research into a solution. The brittleness I'm talking about is not about links that do not point to relevant geometry any longer. I regard that as a logical consequence of links. In my humble opinion copy and track has similar problems but then the reverse: What you currently see, is that because the original has changed or is it the outdated version because the original happens to be moved or something? I would prefer the semantics of the link that simply says the requested geometry is not available.

For a little further thought - I think part of the issue is how FreeCAD lacks some core structure to how the file interoperability is managed. There's very little description of what a file is meant to be, and as a result, an issue in determining what the best way to structure a project or part should be. Some general practice has a part per file, except in the case of assemblies, where part links are grouped into a single .fcstd file. If we decide to describe variant parts/assemblies as their own distinct file standard of file structure, maybe even with their distinctive name, .fcprt and .fcasm, then it acts as a way to separate the concerns of management of these complicated systems from such a generic standard of file interaction.

In that case, I could see .fcprt making good use of an exposed property system and for any .fcasm to be responsible for maintaining a link to the .fcprt. Still, I don't see a good way to get away from copying, here. If we had an extrusion.fcprt and a extrusion_frame.fcasm, then the ideal would be to expose the 'length' property of the .fcprt, and to have it set within the .fcasm. This puts the burden on the assembly to maintain the variant, as opposed to the variant part file. For the amount of different ways to define that length, it would be cumbersome to have those definitions/their linkage tracked within the file. But still, we would need the .fcasm to keep a local object copy for each extrusion body of each length definition.

The best idea I can come up with for the copy flow here would be that the assembly file on any re-compute first checks whether the link can be found to the variant part. If it's not there, we won't pull any updates, and will just rely on our local copy for the rest of the re-compute. If the link is available, and the file is open (or perhaps we create a way to pull relevant file information from the part file without having to have the file open [oh no, temporary files..]), then we check that object's compute UUID to check whether or not we need to copy anything in from the variant part to the assembly file.

I also think if we have a .fcasm and .fcprt file communications standard, then it will make it a lot easier to pull these properties without having to keep every single file open all of the time, and easier manage the properties. An example: a single panel.fcprt stands on four standoff.fcprt in each corner, arranged in the supported_panel.fcasm file. We conclude that the standoff.fcprt 'length' parameter should be driven by the panel.fcprt length parameter. This is a genuine use-case, as we could easily conclude that we'd want the corner mounts of our panel to be larger as the length of the panel increases. In this case, perhaps we need to conclude that we should be able to have each 'exposed' property in the .fcprt act as having both driving/driven behaviour. This way, we would be able to allow the panel.fcprt to describe its 'minimum_length' property to '250mm', create a 'panel_length_addon' property in supported_panel.fcasm set to '150mm', and use that to drive the panel.fcprt 'length' property as the 'minimum_length + panel_length_addon'. This works quite well, as the variant part suggests it requires some input from the inheriting assembly. We can then pass that 'length' back to the parent, and have it drive the standoff.fcprt as having a 'minimum_length' of '10mm', and a 'length' of 'minimum_length * panel_length / 10'.

Right, there was a whole issue on file name extensions and different types of files but I can't find it anymore. This is way beyond this proposal :slightly_smiling_face:. Regarding the example above, I understand what you mean, but I think there are better ways to solve this. In the end supported_panel.fcasm has knowledge about both of the parts. Related to this is the text in #12135 that describes VarSet Relations.

Again, thank you for your time. I think you've given this a lot of thought and that you're definitely the right person to handle this system change, I do wonder if there'll be some unavoidable bumps though. I'd also be interested in helping with the UI for the changes (though I'm not looking for money).

Thanks for the support! I wonder as well, but if they're unavoidable I will run into them anyhow :slightly_smiling_face:

[NomAnor]

It follows the familiar pattern of working around FreeCAD's limitations to acquire powerful features, see below for an explanation.

• A DAG is something that simply disallows cycles, so you can't get into infinite loops when recomputing, so historically this might have been the most logical solution, not anticipating that it may at some point in the future be a limitation for specific use-cases.

• Properties are seen as to drive changes in geometry in the document objects. Since the new shapes have to be computed in the document objects, that was deemed the right granularity for dependencies.

• It may have to do with spreadsheets that can contain values but are not strictly represented by properties, so the easy way out is to do recompute based on objects.

Reading trough the answers, it looks like there might be some downsides in the current core implementation for recompute and dependency tracking. I had similar thoughts about the current recompute approach, but could not find a good documentation how it actually works. Could this be an opportunity to create a new issue regarding the dependency tracking and recompute behaviour? Splitting up the recompute/execute code into separate steps would also belong in there.

Maybe after some discussion in that issue, a clearer solution for this idea will present itself.

I'm not sure if I understood your discussion above. The Idea seems to be to have a read-only copy of the Body feature tree instead of just a variant TopoShape? I think this is more complicated because there is currently no good explanation for what a "part" is. What I've learned so far is that many users think that an App::Part should represent a part and PartDesign::Body represents a body of a part (that is a solid shape). But for generallity PartDesign is not the only workbench to create bodies. So a variant of a part should work with all ways to create a part, not only an App::Part with a single PartDesing::Body in it.

What are the properties/interfaces that a "part" must implement and as such both App::Part and a part variant must provide? Is it just a TopoShape (maybe a compound of the bodies) or a list of bodies (each a TopoShape)?

For debugging purposes a read-only view of the variant body feature tree would be nice, but such a read-only view capability must then be implemented into every object that could be used inside a part. A simple copy would be the easiest way to archive that but there is currently no way to make that copy read-only.

But the idea is to have a general ability to parametrize document objects so the idea of a Shape that represents the variant is wrong because not all objects need to have a Shape. We are still talking about a CAD progam so only being able to parametrise Shape-having objects might be ok. I'm only using PartDesign and Part so I don't know what opportunities exposed properties might provide for other workbenches.

[pieterhijma]

It follows the familiar pattern of working around FreeCAD's limitations to acquire powerful features, see below for an explanation.

• A DAG is something that simply disallows cycles, so you can't get into infinite loops when recomputing, so historically this might have been the most logical solution, not anticipating that it may at some point in the future be a limitation for specific use-cases.
• Properties are seen as to drive changes in geometry in the document objects. Since the new shapes have to be computed in the document objects, that was deemed the right granularity for dependencies.
• It may have to do with spreadsheets that can contain values but are not strictly represented by properties, so the easy way out is to do recompute based on objects.

Reading trough the answers, it looks like there might be some downsides in the current core implementation for recompute and dependency tracking. I had similar thoughts about the current recompute approach, but could not find a good documentation how it actually works. Could this be an opportunity to create a new issue regarding the dependency tracking and recompute behaviour? Splitting up the recompute/execute code into separate steps would also belong in there.

Maybe after some discussion in that issue, a clearer solution for this idea will present itself.

There are already longstanding issues for that, for instance #8059 and #5948. I think the situation regarding performance has been improved because it is now possible to skip some recomputes. I think that there are currently not enough concrete benefits to improve it compared to the risks. In this proposal I may come to the conclusion that it is necessary, contributing to the concrete benefits.

I'm not sure if I understood your discussion above. The Idea seems to be to have a read-only copy of the Body feature tree instead of just a variant TopoShape? I think this is more complicated because there is currently no good explanation for what a "part" is. What I've learned so far is that many users think that an App::Part should represent a part and PartDesign::Body represents a body of a part (that is a solid shape). But for generallity PartDesign is not the only workbench to create bodies. So a variant of a part should work with all ways to create a part, not only an App::Part with a single PartDesing::Body in it.

What are the properties/interfaces that a "part" must implement and as such both App::Part and a part variant must provide? Is it just a TopoShape (maybe a compound of the bodies) or a list of bodies (each a TopoShape)?

For debugging purposes a read-only view of the variant body feature tree would be nice, but such a read-only view capability must then be implemented into every object that could be used inside a part. A simple copy would be the easiest way to archive that but there is currently no way to make that copy read-only.

But the idea is to have a general ability to parametrize document objects so the idea of a Shape that represents the variant is wrong because not all objects need to have a Shape. We are still talking about a CAD progam so only being able to parametrise Shape-having objects might be ok. I'm only using PartDesign and Part so I don't know what opportunities exposed properties might provide for other workbenches.

I think the essence is a difference in approach not relying on links but copying geometry to modify it for a variant, where links are deemed brittle because they can be broken. This looks like what is already implemented in tracking copy-on-change links but has the added benefit that properties can be changed from the outside by means of a special property that links to a property container inside the part. Now that I think of it, this seems like the Assembly 4 approach without temporary documents but with tracking copy-on-change links. Unfortunately, this doesn't generalize exposing properties and is in my opinion working around FreeCAD's limitations to acquire powerful functionality.

What I propose is a change in FreeCAD that acknowledges that document objects in general may want to be parameterized without having to touch the design itself, injecting values of properties to compute a new variant. To do that without having to create a copy (which is currently the way to parameterize in copy-on-change), my idea is to split the execution steps to make sure we can compute shapes by presenting it with a different context of properties and these properties override the original ones.

Regarding App::Part I think it is nothing more than a group of document objects that we name App::Part with some additional features such as setting the material. We consider it the basis for assembly, so sub assemblies should also be App::Part.

At least this is currently my view on it. Please correct me if I'm wrong anywhere.

[chennes]

Hello @pieterhijma -

Thank you again for your grant proposal. It is clear that you have put extensive thought into this complex issue, and your thorough responses above are much appreciated. Because of the complex nature of this project, the FPA feels that more community discussion is required before it is ready for funding, so we are declining the grant at this time. This is not a final say on the issue, and you are welcome to resubmit the proposal in the future, once the outstanding discussions have been settled.

[prokoudine]

@chennes Do either the grant review committee or the FPA general assembly (or both) have a suggestion for where this further conversation should take place and what the process would look like?

[yorikvanhavre]

@prokoudine : @pieterhijma and me scheduled a meeting tomorrow to start attacking the matter https://github.com/FreeCAD/FreeCAD/pull/12532#issuecomment-2060704271 - anyone welcome! I propose we start from there to define the next steps

[pieterhijma]

As Yorik mentioned in the comment above, we had a meeting about PR #12532 and this grant proposal. I provided a summary on the part regarding the PR in this comment. I will try to summarize the meeting regarding the grant proposal here.

Yorik and Shai were present. Thank you both for your interest!

From the meeting and the discussion here in this issue, it is clear that this subject is complex. The problems with the current situation are not very clear, let alone the solution.

One of the main points of Yorik is that FreeCAD is community driven and that we should strive for the core logic of FreeCAD to be accessible to the greater community. Since this matter is complex, it has a likely chance that the core becomes even more complex with even less people that understand it. Especially when we start talking about changing the execution model. That would be a large change and I agree that there is chance that the core becomes more complex and I also agree that this is an unwanted situation.

Another issue that we discussed is the problem with dependency cycles. From Yorik's point of view, the DAG (directed acyclic graph) of document objects is one of the backbones of FreeCAD that allows FreeCAD to do efficent recomputes. So, we should be very careful with that and preferably not change it.

Again, I must agree that it is good to be able to rely on the DAG of document objects, but from my point of view, we also have to acknowledge that hidden references are there for a reason and that sometimes we want to be more flexible than what the current DAG and recompute logic offers us. Hidden references are necessary for configurations and the current implementation of exposing variable sets makes use of them. Additionally, in my humble opinion it is quite easy for users to create cyclic dependencies that are difficult to debug for regular users. Moreover, hidden references may require dependencies to recompute, so the core has actually multiple dependency checking systems.

Yorik and Shai made clear that they are in support of the overall idea of exposing properties to create variant parts. And for moving forward, they had some good suggestions to make sure the developments are accessible to the community. Yorik suggests to split up the work in smaller PRs that allows the community to understand the changes in a more incremental way. From his point of view the above can already be split in seperate concerns, for example:

• making it possible to refer to properties from parent document objects
• make variants from these kinds of document objects

Another suggestion was to make it more clear that this is more a research project that tries to find solutions to acquire the functionality of variant parts, then report on the various options available and define clear decision moments in which the community can decide which way to go.

The current idea is to close this issue and create another grant proposal that incorporates this feedback. I'm very happy that Yorik and Shai were willing to share their view and I can absolutely see their point. So, I'm going to rethink this proposal incorporating this feedback and resubmit.

@yorikvanhavre and @shaise, please comment if I misrepresent anything or if I miss something.

[yorikvanhavre]

This resumes well what was discussed during our meeting. Indeed we have no concerns about these two goals, we had concerns about the unclear and possibly unwanted consequences for FreeCAD.

Going against the DAG is indeed for me something highly delicate. This definitely deserves a very cautious discussion an even more cautious plan, and probably a good analysis of the current hidden ref feature, to know if it is solid enough to bear more weight.

Resubmitting a grant proposal that is more clearly a "research program" is IMHO totally valid. GSoC also permits those, and the benefits are clearly there.

[pieterhijma]

I've just submitted #10 as a follow up that incorporates the feedback.