as a user I'd like to be able to save scenarios and be able to load them with a record of all their applied projects
Background
MetCouncil would just pickle the scenario, but for some reason some thing (specifically lambda functions) are "unpicklable". It's not entirely clear why lambda functions would need to be pickled, but suspect it is b/c the validation requirements are within the network object instances.
Resolution Options
Pickle vs YAML vs networks-only
Store scenarios w/out networks as pickles myscen.pickle()myscen = load_pickle()
Store scenarios as a human-readable toml/yaml, mapping to stored networks and project card filenames myscen.write()myscen = load_scenario()
Don't worry about scenarios, just add a variable to networks themselves that store the applied project names net.projects and add a YAML file to network output specifying applied projects
Where store project data
Writing out projectcards to files in subfolder
Add project card data to network YAML
Just rely on project names
Describe the bug
Rachel likes to save scenario objects out as pickle files via pickle.dump(version_00_scenario, open(working_scenario_filename, 'wb')), and load them via pickle.load(open(working_scenario_filename, 'rb')). This allows her restart from a previous scenario pickle file and not need to reapply previous project cards. The benefit of having the scenario saved is also that it is easy to check what project cards have been applied via scenario.applied_projects.
With the new Network Wrangler, we currently cannot use pickling. When running pickle.dump(), we are getting:
as a user I'd like to be able to save scenarios and be able to load them with a record of all their applied projects
Background
MetCouncil would just pickle the scenario, but for some reason some thing (specifically lambda functions) are "unpicklable". It's not entirely clear why lambda functions would need to be pickled, but suspect it is b/c the validation requirements are within the network object instances.
Resolution Options
Pickle vs YAML vs networks-only
myscen.pickle()
myscen = load_pickle()
myscen.write()
myscen = load_scenario()
net.projects
and add a YAML file to network output specifying applied projectsWhere store project data
Describe the bug
Rachel likes to save scenario objects out as pickle files via
pickle.dump(version_00_scenario, open(working_scenario_filename, 'wb'))
, and load them viapickle.load(open(working_scenario_filename, 'rb'))
. This allows her restart from a previous scenario pickle file and not need to reapply previous project cards. The benefit of having the scenario saved is also that it is easy to check what project cards have been applied viascenario.applied_projects
.With the new Network Wrangler, we currently cannot use pickling. When running pickle.dump(), we are getting:
@e-lo What would be an alternative to the scenario pickle?
To Reproduce
Steps to reproduce the behavior:
Failing tests
Triggering line of code
Thoughts on resolution
Full stack trace
Environment
Operating system: Context (conda, jupyter, etc): Environment (e.g. output from
conda list
):