Fix search of success files in FileSource

twitter / scalding

A Scala API for Cascading

http://twitter.com/scalding

Apache License 2.0

3.48k stars 704 forks source link

Fix search of success files in FileSource #1862

Closed dieu closed 6 years ago

dieu commented 6 years ago

Currently, we do sumByKey on usedDirs which has type Iterable[(String, (String, OrVal, OrVal))], what ends up with summing of strings by monoid, what's unnecessary and waste a lot of time concatenation of strings.

gerashegalov commented 6 years ago

Thanks for fixing it @dieu !