Closed reelmatt closed 4 years ago
@reddigari, your method was exactly what I had in mind and is what I was attempting to do. I made the constant name DEFAULT_OPTIONS
with the thinking we might want something like DEFAULT_STYLES
as well (color, port shapes/kinds?). Would love to hear what others think in terms of this approach and how it might scale/adapt going forward.
One example that might guide some thinking: I realized storing options in a dictionary means we can pass them in as **kwargs
to a pandas call. However, instead of the file
option that was at the IONode
level, the two concrete nodes now have custom filepath_or_buffer
and path_or_buf
options to mirror the pandas function. This seams simpler for node execution, but might require more custom code for file handling/parsing?
@reelmatt Definitely agree that passing the options as **kwargs
is a good goal, but not necessary if it ends up being a hassle. One strategy I've used when wrangling keyword argument dicts is to extract the problem ones, pass them as positional or explicit keyword arguments, and then pass the remaining dict as kwargs:
file_buf = options.pop("file")
arg_name = options.pop("invalid_arg")
pandas.whatever(file_buf, arg_name=arg_name, **options)
A few change highlights:
add_node
method toupdate_or_add_node
options
dictionary. Idea is this can store Node parameters (e.g. 'delimiter' for theReadCsvNode
. The idea is these could be passed in to whatever pandas function needs them, and are also used to feed front-end for modifying behavior.Added some new Postman tests to hit new functionality. There may still be some wonkiness and
KeyErrors
, but most of it should be working.