EntilZha / PyFunctional

Python library for creating data pipelines with chain functional programming
http://pyfunctional.pedro.ai
MIT License
2.41k stars 132 forks source link

Wierd bug with flatten and partition #124

Closed cdoool closed 6 years ago

cdoool commented 6 years ago

Hi! First of all, thanks for the great library. Thanks to it I no longer want to migrate back to Scala.

On a first day of using it, I encountered a strange bug: val = seq([1,2,3,4,5,6]).partition(lambda x:x>2) (val = [[3, 4, 5, 6], [1, 2]]) But val = seq([[1,2,3],[4,5,6]]).flatten().partition(lambda x:x>2) (val = [[3, 4, 5, 6], []])

Surely, it's a bug?

EntilZha commented 6 years ago

Thanks @cdoool for opening an issue! I agree that the behavior seems wrong. I did a little more debugging and I think this example shows that it is due to the sequence being consumed twice in partition. The culprit is that there are two filter calls in the implementation https://github.com/EntilZha/PyFunctional/blob/master/functional/transformations.py#L672. It shouldn't be too hard to rewrite that to do only a single pass/consume the sequence once. I could probably have a fix this weekend and make use of the fancy new travis auto publish to pypi to get the fix out shortly after.

Thanks for the bug report!

In [36]: a, b = seq([[1, 2, 3], [4, 5, 6]]).flatten().partition(lambda x: x > 2)
In [37]: a
Out[37]: [3, 4, 5, 6]
In [38]: b
Out[38]: []
In [39]: a, b = seq([[1, 2, 3], [4, 5, 6]]).flatten().partition(lambda x: x > 2)
In [40]: b
Out[40]: [1, 2]
In [41]: a
Out[41]: [] 
EntilZha commented 6 years ago

@cdoool I believe I've fixed the issue on master. I included the example above in the tests, but it would be great if you can verify for your specific use case that it works that would be great. If everything looks good I can push out a minor release on PyPI

cdoool commented 6 years ago

Can confirm everything works fine now. Many thanks!

EntilZha commented 6 years ago

Thanks, new version should be out on pypi now