PyWorkflowApp / visual-programming

A Python Visual Programming Workspace for Data Science
MIT License
33 stars 12 forks source link

Update Node route, new Node classes, altered POST data handling #24

Closed reelmatt closed 4 years ago

reelmatt commented 4 years ago

A few change highlights:

Added some new Postman tests to hit new functionality. There may still be some wonkiness and KeyErrors, but most of it should be working.

reelmatt commented 4 years ago

@reddigari, your method was exactly what I had in mind and is what I was attempting to do. I made the constant name DEFAULT_OPTIONS with the thinking we might want something like DEFAULT_STYLES as well (color, port shapes/kinds?). Would love to hear what others think in terms of this approach and how it might scale/adapt going forward.

One example that might guide some thinking: I realized storing options in a dictionary means we can pass them in as **kwargs to a pandas call. However, instead of the file option that was at the IONode level, the two concrete nodes now have custom filepath_or_buffer and path_or_buf options to mirror the pandas function. This seams simpler for node execution, but might require more custom code for file handling/parsing?

reddigari commented 4 years ago

@reelmatt Definitely agree that passing the options as **kwargs is a good goal, but not necessary if it ends up being a hassle. One strategy I've used when wrangling keyword argument dicts is to extract the problem ones, pass them as positional or explicit keyword arguments, and then pass the remaining dict as kwargs:

file_buf = options.pop("file")
arg_name = options.pop("invalid_arg")
pandas.whatever(file_buf, arg_name=arg_name, **options)