nessos / Streams

A lightweight F#/C# library for efficient functional-style pipelines on streams of data.
http://nessos.github.io/Streams/
Other
382 stars 46 forks source link

ParStream.foldBy do not preserve order when requested to #9

Closed mexx closed 9 years ago

mexx commented 9 years ago

When following pipeline is processed

let map =
[(1, 1);(2, 2);(2, 3);(1, 4)]
|> ParStream.ofSeq
|> ParStream.ordered
|> ParStream.foldBy fst (fun l x -> x :: l) (fun x _ -> x) (fun () -> [])
|> ParStream.toSeq
|> Map.ofSeq
|> Map.map (fun _ -> Seq.map snd)

The results are unstable, sometimes I get:

> map.[1];;
val it : seq<int> = seq [4; 1]
> map.[2];;
val it : seq<int> = seq [2; 3]

On some other evaluation I get:

> map.[1];;
val it : seq<int> = seq [4; 1]
> map.[2];;
val it : seq<int> = seq [3; 2]

With given pipeline I would expect to always get the second version.

palladin commented 9 years ago

Yep it is a bug and in comparison with PSeq let map' () = [(1, 1);(2, 2);(2, 3);(1, 4)] |> PSeq.ordered |> PSeq.groupBy fst |> Map.ofSeq |> Map.map (fun _ -> Seq.map snd)

the order of values must be preserved.

palladin commented 9 years ago

Fixed in https://github.com/nessos/Streams/commit/b8a807710249fe4c83a7f3b2440ab8fc078f50da