[minor] Use factories in existing nodes

liamhuber commented 2 months ago

This leverages the @classfactory tools from the last PR to make existing Function and Macro nodes pickleable. That means that the entire standard (need to check for- and while- loops still) ecosystem can now be pickled:

import pickle

from pyiron_workflow import Workflow

@Workflow.wrap.as_macro_node("plus_3")
def AddThree(self, x):
    self.one = x + 1
    self.two = self.one + 1
    self.three = self.two + 1
    return self.three

def halve(x):
    return 0.5 * x

wf = Workflow("in_vinegar")
wf.x = Workflow.create.standard.UserInput(42)
wf.p3 = AddThree(wf.x)
wf.h = Workflow.create.function_node(halve, wf.p3, output_labels="halved")
wf()

reloaded = pickle.loads(pickle.dumps(wf))
print(wf.outputs.to_value_dict(), reloaded.outputs.to_value_dict())

This had one giant, nasty side effect: storing with the h5io back-end is almost completely useless. h5io can't handle anything that returns a non-standard reconstructor in its reduced value, which is now almost all of the usual nodes. For what I suspect is a similar reason, we can't just slap a node onto a DataContainer and to-hdf it.

To mitigate this pain, I switched the default back-end over to tinybase (which works fine, I believe due to its fallback onto cloudpickle -- kudos to @pmrv), and simply pickle the input node on the NodeOutputJob interface with pyiron_base. I'll also stack on top of this a pickle (and probably cloudpickle, in case the nodes carry unpickleable data) back-end and make that default. Hierarchical and browsable long-term storage is still a key objective (👍 @niklassiemer), but I want to delay it until we have a storage tool that actually supports such storage in a pickle-compliant way. I.e. I want to start with a foundation of being pickle-compliant, and then find (or more likely write) a storage tool that is also pickle-compliant, than to bend over backwards to get long-term storage now.

github-actions[bot] commented 2 months ago

:point_left: Launch a binder notebook on branch _pyiron/pyiron_workflow/use_factories_in_existingnodes

codacy-production[bot] commented 2 months ago

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation	Diff coverage
:white_check_mark: +0.23% (target: -1.00%)	:white_check_mark: 98.67%

Coverage variation details

| | Coverable lines | Covered lines | Coverage | | ------------- | ------------- | ------------- | ------------- | | Common ancestor commit (37fd191ff8009793bd7bdaa10b020d9213d23468) | 3698 | 3269 | 88.40% | | | Head commit (d620c7be55b2656d1e9ce677b67d9a9cde04ca31) | 3728 (+30) | 3304 (+35) | 88.63% (**+0.23%**) | **Coverage variation** is the difference between the coverage for the head and common ancestor commits of the pull request branch: ` - `

Diff coverage details

| | Coverable lines | Covered lines | Diff coverage | | ------------- | ------------- | ------------- | ------------- | | Pull request (#303) | 75 | 74 | **98.67%** | **Diff coverage** is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: `/ * 100%`

See your quality gate settings Change summary preferences

_{Codacy will stop sending the deprecated coverage status from June 5th, 2024. Learn more}

coveralls commented 2 months ago

Pull Request Test Coverage Report for Build 8886092588

Details

0 of 0 changed or added relevant lines in 0 files are covered.
45 unchanged lines in 9 files lost coverage.
Overall coverage increased (+0.2%) to 88.627%

Files with Coverage Reduction	New Missed Lines	%
function.py	1	98.11%
injection.py	1	93.28%
storage.py	1	92.31%
snippets/factory.py	3	97.76%
io_preview.py	6	95.83%
node.py	7	94.36%
macro.py	7	92.09%
job.py	9	91.96%
node_library/standard.py	10	93.15%
<!--	Total:	45	-->

Totals
Change from base Build 8883760424:	0.2%
Covered Lines:	3304
Relevant Lines:	3728

pyiron / pyiron_workflow