Improve compilation latency for non-inferable case

[tkf]

The following snippets shows a big jump in the compilation latency after eb430cf76a5c25aa909858c20424aead21cfecfd (#403):

using DataFrames
using Glob
using Pkg: TOML
using Transducers

function asnamedtuple(d)
    kvs = [(Symbol(k) => v) for (k, v) in pairs(d)]
    sort!(kvs; by = first)
    return (; kvs...)
end

demo(depot_path = DEPOT_PATH) =
    depot_path |>
    MapCat() do path
        readdir(glob"registries/*/*/*/Package.toml", path)
    end |>
    Map() do path
        p = TOML.parsefile(path)
        get!(p, "subdir", nothing)
        p["datadir"] = dirname(path)
        depsfile = joinpath(p["datadir"], "Deps.toml")
        if !isfile(depsfile)
            p["deps"] = nothing
            return p
        end
        p["deps"] =
            values(TOML.parsefile(depsfile)) |>
            MapCat(pairs) |>
            MapSplat(tuple) |>
            Map(NamedTuple{(:name, :uuid)}) |>
            Set
        p
    end |>
    Map(asnamedtuple) |>
    x -> copy(DataFrame, x)

@time demo()

Transducers.jl commit	w/ Julia 1.6.0-DEV.536	w/ Julia 1.5	w/ Julia 1.4
65db24b0e874a4472eb3b81543c11306c2fd8581 (pre #403)	4.6 sec	3.6 sec	5.0 sec
eb430cf76a5c25aa909858c20424aead21cfecfd (#403)	17 sec	77 sec	24 sec
29d65cb43cdc1297e624c1372bbe80373d93b483 (This PR)	8.7 sec	27 sec	11 sec

Commit Message

Improve compilation latency for non-inferable case (#412)

The tail-call pattern introduced in eb430cf76a5c25aa909858c20424aead21cfecfd (#403) for linear indexing arrays seem to invoke a large compiler latency (see #412 for an example). This patch tries to mitigate the problem by introducing a different more dynamic code path for __foldl__ on arrays when the reducing function is not inferrable.

[codecov[bot]]

Codecov Report

Merging #412 into master will increase coverage by 1.22%. The diff coverage is 75.00%.

@@            Coverage Diff             @@
##           master     #412      +/-   ##
==========================================
+ Coverage   89.52%   90.75%   +1.22%     
==========================================
  Files          25       25              
  Lines        1585     1600      +15     
==========================================
+ Hits         1419     1452      +33     
+ Misses        166      148      -18     

Flag	Coverage Δ
#unittests	90.75% <75.00%> (+1.22%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/basics.jl	90.00% <50.00%> (+8.42%)	:arrow_up:
src/processes.jl	92.68% <83.33%> (-0.36%)	:arrow_down:
src/library.jl	93.15% <0.00%> (-0.28%)	:arrow_down:
src/show.jl	89.51% <0.00%> (+0.07%)	:arrow_up:
src/unordered.jl	96.61% <0.00%> (+0.31%)	:arrow_up:
src/progress.jl	89.79% <0.00%> (+1.02%)	:arrow_up:
src/core.jl	85.56% <0.00%> (+1.06%)	:arrow_up:
src/groupby.jl	82.14% <0.00%> (+5.77%)	:arrow_up:
src/reduce.jl	84.55% <0.00%> (+9.14%)	:arrow_up:
... and 1 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 1f67507...29d65cb. Read the comment docs.