Need a setting for custom nodes to force execution on re-run

[andydandy74]

This is kind of related to DynamoDS/Dynamo#3273, but I am creating a separate issue for it because this is a request for an entirely different feature.

The Dynamo engine is designed in such a way that it will only re-execute those parts of a graph that have been subjected to a change (e.g. by changing an input value, by newly connected nodes or by changes in the Revit model). I think everyone agrees that this, in principle, is a very, very clever thing as it significantly reduces processing time in subsequent runs of a graph. However, this mechanism also has one major shortcoming: Python-based custom nodes that interact with the Revit API will not get re-executed when changes to the Revit model have been made. I am thinking in particular of custom nodes that act as element collectors. All the major Python-based packages contain these kinds of nodes and they practically become useless after the first run of a graph as they cannot react on model changes anymore without re-opening the graph first - a significant impedement to a lean workflow.

My suggestion is to include an "Always re-execute" setting in the custom node properties in the form of a checkbox (set to False by default). Custom nodes that have that property set to True would be executed on each run of a graph.

[dimven]

:+1:

[jnealb]

ditto

[eproca]

It is clear that some times it is convenient to have the full re-execution of the graphic. But I really hope that we don't come back to have always the full re-execution. The actual system makes an amazing difference in speed and fluidity of working when you are making heavy modelling or anything that involves some times to process

Maybe an easy to implement solution is just to be able to choose if you want the full re-execution or the current system. I mean to define it globally for the whole graphic and not node by node.

[andydandy74]

@eproca - I think it's safe to say that nobody wants to go back to full execution on each run of a graph. But re-executing the entire graph because it contains 2 or 3 Python-based nodes that just don't know that they should re-execute is a little bit over the top, in my opinion, like using a sledgehammer to crack a nut.

I am not proposing that the user of a graph has to determine which custom nodes need constant re-execution. I see this as a tool for package developers to improve UX.

[josols]

+1

[kronz]

Great discussion. How do you all feel about the method that Nate Miller uses with the Lunchbox tools, with a "toggle" input, when the user changes a boolean toggle, the node re-executes.

[andydandy74]

@kronz - I would say its a valid workaround. But it leaves the responsibility with the user, not the developer. Plus, it's an additional input which I'd like to avoid.

You mentioning it did however inspire another thought. Currently, all nodes that generate a random number do not get re-executed either (bug or feature?). But if they did, we could include one of them in a Python-based custom node, wire it into the Python node (without actually having the Python node perform an operation on that number) and thus force the Python node into re-execution. This could provide a low-impact solution to the problem but it would mean that the following two criteria need to be met:

• Nodes that generate random numbers get re-evaluated on each run of a graph
• Custom nodes that contain elements that are re-evaluated are re-evaluated as well

[andydandy74]

BTW: Another candidate would be the DateTime.Now node - but that one also doesn't re-execute. And that definitely looks like a bug to me. "Now" should really mean "now" every time I run a graph.

[persp]

@andydandy74 : I have sufferings of simular problems now. Do you mean that it's all about the python scripts in the graph and there is no solution but to wait with impatience before develper fix it?

[andydandy74]

@persp - the short answer is: Yes. But if you are too impatient you can always use the same method that is used in LunchBox - it's pretty robust. ;-)

[DanRumery]

How about requiring developers to tag such nodes as side-effecting similar to how side-effecting actions are tagged in Haskell with the IO type class. Dynamo would then know to (re)-execute them given these nodes may produce different results on each run. Additionally Dynamo would know to sequence such side-effecting operations in order to ensure well-defined behavior in an otherwise lazy-evaluated expression tree.

[ParametricMonkey]

+1

[ekkonap]

I might recently have stumbled on another workaround. My script to duplicate Sheets evaluates the project for existing sheet numbers to find the first free consecutive sheet number suffix. The sheet is created with a Custom node with some python in it (not sure where I stole that at the moment), and it will make new sheets on every run. In short, if you store some result of a script in the Revit project and then evaluate the project in the next run for that new value or element (Nb only new element confirmed with my script, new value might not work) the change in the project changes the python input, causing it to execute. Can anyone confirm or defute that?

[andydandy74]

@ekkonap - Unfortunately this will only work for cases such as the one you outlined. My original use case for raising this issue were nodes that contain e.g. FilteredElementCollector, so they basically just reads stuff from the model. It's hard to explain to users why they need to toggle a Boolean node each time they re-run a script when the built-in collector nodes work just fine. ;-)

[ekkonap]

@andreas - I completely agree that your proposal would be useful. I just found a workaround tjat is rather clumsy for read-only applications. Good to know that it indeed has limitations if you want to only read from a model, and I can certainly see the need for that.

[DanRumery]

Why is this issue still not resolved? Even in the purest languages (e.g. Haskell) there is still a mechanism for including side-effecting actions in the program. I've found that the 'boolean toggle' workaround is confusing for the average user ("why do I have to toggle it? why doesn't it matter if it's true or false as long as you change it on each run?").

Can we at least give C# developers the ability to tag a node as side-effecting (needing re-evaluation despite no change in input) and the Dynamo engine respect this?

[andydandy74]

@johnpierson - This issue seems to have a fairly good amount of community support. Any particular reason it got closed?

[johnpierson]

I had closed this as it relates to another open issue, https://github.com/DynamoDS/Dynamo/issues/3273

[andydandy74]

Okay, got it.

[johnpierson]

Although, I am now noticing that you referenced it at first and said this was different.

[johnpierson]

Do you feel this is still its own thing that needs focus? Apart from the referenced issue?

[andydandy74]

After re-reading Nate's issue, I would rather have this one kept open as well. I believe it would be computationally more economic to be able to recompute on a per-node basis.

[johnpierson]

That is what I thought. Thanks for the update. :)

[cbonillo91]

Did someone manage to find out a temporary solution for this topic without using the Toogle Boolean solution? In my case I would need to force re-execution of a node in order to make a webservice request, so that the content of an item box updates without having to select another item and come back to the previous one in order to see the changes...

[s-fad]

Looks like* I've done it via read/write current toggle value to the file. Think that it can b optimized by creation/deletion of the flag file without read/write:

I am rather new to dynamo, and just did several successful tests.

[ChrisGithubWork]

Hey at all, this one is open since 2015 and the related ones got closed unresolved DynamoDS/Dynamo#4603 or DynamoDS/Dynamo#6759 due to duplicates or are still open, like for example DynamoDS/Dynamo#3273 .
I think since this is open for several years there wont be an update, or did someone resolve this and didn't update the issues? If no, does someone have another nice workaround for this issue?

[johnpierson]

There are quite a few workarounds with custom packages nowadays:

1. Use SpringsUI "force run" node,
2. Use Data-Shapes Force Run tool in the view extension "Data-Shapes" tab in Dynamo.
3. Build Dynamo from source and use the force run option that is available at that point.

---

Even with those workarounds, this is still an outstanding task for OOTB Dynamo.

[Amoursol]

We will have this functionality enabled when we release the TuneUp extension https://github.com/DynamoDS/TuneUp - You can build from source on this right now as a WIP but we still have a couple of tasks left before this gets wrapped up properly.

[ChrisGithubWork]

Thanks for the replies !

@johnpierson

1. Not working anymore with dyn 2.0 ... see DynamoDS/Dynamo#8943
2. This looks amazing! Seems to work. Will test it now. Thanks!
3. This is not a valid workaround for a user. This would be a valid workaround for a developer.

@Amoursol Thanks for the hint. When do you think is the release?

[Amoursol]

@ChrisGithubWork - We cannot give explicit dates as they are subject to change, but we are targeting it for this current Quarter. First release will be via the Package Manager to garner feedback, then wrapped into Core subsequently.

[HadiMoosavi]

Hello everybody, Trying to see if this issue is resolved. I made a zero-touch node in c# that reads some basic parameter value from a model element. When I change that parameter in the Revit model, my node does feel like a re-run is necessary. Do you still recommend a bool toggle?

[Amoursol]

@HadiMoosavi - You can use this functionality via the TuneUp extension, available on the Package Manager now and functioning in Dynamo 2.5 onwards 😊

Read more about it here: https://dynamobim.org/tuneup-extension-explore-your-node-and-graph-execution-times/