Closed lexming closed 2 years ago
Not sure what happened with Jenkins, but running tox
on my side works fine:
https://gist.github.com/53be3bff4225c96598eb2ddcfe3889a7
what is the full code for your timings? in particular why not use list(itertools.chain(*list_of_all_lists))
instead of using monid at all?
Superseded by https://github.com/hpcugent/vsc-filesystems/pull/84
This PR adds a variation of the
Monoid
class specifically optimized to concatenate lists.Monoid
is currently used by vsc-filesystems to concatenate lists of filesystem/quota attributes and it is painfully slow. To the point where some scripts such asdquota
become unusable.The issue is that the
reduce
method used inMonoid
does not scale. The table below shows the time needed to inject into aMonoidDict
an increasing amount of lists with the current mappend used in vsc-filesystems, some other alternative mappends and the newMonoidConcat
class (withoutreduce
):list.extend
:list.append
:itertools.chain
:reduce