pipe and **kwargs

[dilzeem]

Hi,

I am not sure if this something that should be done, or if this bad practice. But I wanted something like this to work.

import toolz

some_dict = {"hello": "world", "foo": "bar", "number": 0}

def first_func(hello, foo, **kwargs):
    hello = "John"
    foo = "baz"
    kwargs['newvalue'] = True
    return kwargs

def second_func(number, **kwargs):
    number = number + 1
    kwargs['number'] = number
    kwargs['anothervalue'] = False
    return kwargs

running something like this works:

first_func(**some_dict)
>>> 
{'number': 0, 'newvalue': True}

second_func(**some_dict)
>>> 
{'number': 1, 'anothervalue': False}

But this won't work, which I understand why.

new_dict = toolz.pipe(**some_dict, first_func, second_func)

The idea is that I want some_dict to remain untouched, but new_dict to capture relevant information along the way.

Is this bad practice? Or not a relevant usecase? or is there a better way to do this?

[jtwool]

I like this; I'd create a new class to handle this because you only want it to work on the one object.

from copy import copy 

class PipeableDict(dict):

    def pipe(self, fn):
        _copy = copy(self)
        fn(_copy)
        return _copy

    def pipeline(self, *fns):
        _copy = copy(self)
        for fn in fns:
            _copy = _copy.pipe(fn)
        return _copy

if __name__ == "__main__":

    d = PipeableDict({"foo":"abc",
                      "bar": 123})

    e = d.pipe(lambda x:x.update(buzz=3))\
         .pipe(lambda x:x.update(name="John Doe"))\
         .pipe(lambda x:x.update(foo="ABC"))

    print("Pipe")
    print(d)
    print(e)

    f = d.pipeline(
        lambda x:x.update(buzz=3),
        lambda x:x.update(name="John Doe"),
        lambda x:x.update(foo="ABC")
    )

    print("Pipeline")
    print(d)
    print(f)      

Probably needs a better name. Maybe ImmutablePipeableDict?

[dilzeem]

Thanks for the comment. It helped look into this further, and get something that I am okay with for the time being.

I think I found what I was looking for with a minor tweak another issue/feature request. Using the starapply mentioned here: https://github.com/pytoolz/toolz/issues/486

I made a minor tweak, where only key word arguments are allowed.

@curry
def unpack(func, kwargs):
    return func(**kwargs)

new_dict = toolz.pipe(some_dict, unpack(first_func), unpack(second_func))

>>>new_dict
{'newvalue': True, 'number': 1, 'anothervalue': False}

>>>some_dict
{"hello": "world", "foo": "bar", "number": 0}

some_dict remains unchanged.

This is pretty useful for me in running pipelines and tracking what arguments were used. Though the function definitions need to have the **kwargs defined.

And if you decide to use the variable, then it gets removed from **kwargs , but I guess it is nice to explicitly push the required values back into **kwargs.

[mrucci]

Is this bad practice? Or not a relevant usecase? or is there a better way to do this?

I would personally find the behaviour pretty obscure and would expect the resulting implementation to be brittle since it depends on the number and order of argument in each function in the pipe.

I've never used it before but the problem you are solving reminds me of passports, where the basic idea is that you pass around an object (the passport) and each function in the pipe can stamp it with the relevant information.

Explicit is better than implicit.

[dilzeem]

thanks for the comment and the link. I will check it out. it's nice to get some feedback here.

to clarify the order arguments don't need to be in a certain order for this solution. they just have to present in the dictionary passed to the function.