timholy
commented
4 years ago I've not had any luck either, but for your reference here's a snapshot of how I go about this. While it's not commonly done this way, I personally prefer to write out the precompile directives by hand so that I can leverage my understanding of the package internals to control stuff like coverage of array eltypes and dimensionality. (In other words, I don't use parcel or write in most cases.) Here's a session:
julia> using ImageQualityIndexes, TestImages, SnoopCompile
julia> img_gray_1, img_gray_2 = testimage("fabio_gray_512"), testimage("cameraman");
julia> tinf = @snoopi tmin=0.01 assess_ssim(img_gray_1, img_gray_2)
8-element Vector{Tuple{Float64,Core.MethodInstance}}:
(0.014342784881591797, MethodInstance for copy(::Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple},Nothing,typeof(ImageCore.channelview),Tuple{NTuple{6,Matrix{ColorTypes.Gray{Float64}}}}}))
(0.015332937240600586, MethodInstance for mean(::Matrix{Float64}))
(0.02500009536743164, MethodInstance for materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(-),Tuple{SubArray{ColorTypes.Gray{Float64},2,Matrix{ColorTypes.Gray{Float64}},Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},false},Matrix{ColorTypes.Gray{Float64}}}}))
(0.029573917388916016, MethodInstance for imfilter(::Type{ColorTypes.Gray{Float64}}, ::Matrix{ColorTypes.Gray{Float32}}, ::Tuple{ImageFiltering.KernelFactors.ReshapedOneD{Float64,2,0,OffsetArrays.OffsetVector{Float64,Vector{Float64}}},ImageFiltering.KernelFactors.ReshapedOneD{Float64,2,1,OffsetArrays.OffsetVector{Float64,Vector{Float64}}}}, ::String))
(0.042634010314941406, MethodInstance for materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(ImageQualityIndexes._mul),Tuple{SubArray{ColorTypes.Gray{Float64},2,Matrix{ColorTypes.Gray{Float64}},Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},false},SubArray{ColorTypes.Gray{Float64},2,Matrix{ColorTypes.Gray{Float64}},Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},false}}}))
(0.1404099464416504, MethodInstance for materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(/),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(*),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(+),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(*),Tuple{Int64,Base.ReinterpretArray{Float64,2,ColorTypes.Gray{Float64},Matrix{ColorTypes.Gray{Float64}}}}},Float64}},Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(+),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(*),Tuple{Int64,Base.ReinterpretArray{Float64,2,ColorTypes.Gray{Float64},Matrix{ColorTypes.Gray{Float64}}}}},Float64}}}},Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(*),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(+),Tuple{Base.ReinterpretArray{Float64,2,ColorTypes.Gray{Float64},Matrix{ColorTypes.Gray{Float64}}},Base.ReinterpretArray{Float64,2,ColorTypes.Gray{Float64},Matrix{ColorTypes.Gray{Float64}}},Float64}},Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(+),Tuple{Base.ReinterpretArray{Float64,2,ColorTypes.Gray{Float64},Matrix{ColorTypes.Gray{Float64}}},Base.ReinterpretArray{Float64,2,ColorTypes.Gray{Float64},Matrix{ColorTypes.Gray{Float64}}},Float64}}}}}}))
(0.35606908798217773, MethodInstance for imfilter(::Type{ColorTypes.Gray{Float64}}, ::MappedArrays.MappedArray{ColorTypes.Gray{Float32},2,Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}},MappedArrays.var"#2#4"{ColorTypes.Gray{Float32}},MappedArrays.var"#3#5"{ColorTypes.Gray{FixedPointNumbers.N0f8}}}, ::Tuple{ImageFiltering.KernelFactors.ReshapedOneD{Float64,2,0,OffsetArrays.OffsetVector{Float64,Vector{Float64}}},ImageFiltering.KernelFactors.ReshapedOneD{Float64,2,1,OffsetArrays.OffsetVector{Float64,Vector{Float64}}}}, ::String))
(0.6890788078308105, MethodInstance for assess_ssim(::Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}, ::Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}))I usually cherry-pick the juicy targets and put things into their corresponding packages. In this case, your most expensive MethodInstance to infer is the assess_ssim call at nearly 0.7s. Also note that the types are not too complicated, we can imagine this being sufficiently broadly useful that it's worth precompiling. So then I add this to your package definition:
function _precompile_()
ccall(:jl_generating_output, Cint, ()) == 1 || return nothing
eltypes = (N0f8, N0f16, Float32, Float64) # eltypes of parametric colors
pctypes = (Gray, RGB) # parametric colors
cctypes = (Gray24, RGB24) # non-parametric colors
dims = (1, 2, 3)
for n in dims
for C in pctypes, T in eltypes
precompile(assess_ssim, (Array{C{T},n}, Array{C{T},n}))
end
for C in cctypes
precompile(assess_ssim, (Array{C,n}, Array{C,n}))
end
end
end
VERSION >= v"1.4.2" && _precompile_()Then I quit Julia and run the above again. When I do so, sadness strikes: the assess_ssim call is still there, which means precompilation didn't "take."
So now it's time to investigate why. I already have a strong guess (I bet we'll have to ensure that imfilter precompiles), but to see whether additional efforts are likely to have any success let's first check out whether we have rampant invalidation; if we do, nothing will help and all our efforts will be a waste. So start a new session (must be Julia's master branch) and do this:
julia> using SnoopCompileCore
julia> invs = @snoopr using ImageQualityIndexes;
julia> using SnoopCompile
julia> length(uinvalidated(invs))
288That's around where we are with ImageCore. If you do
julia> trees = invalidation_trees(invs)
14-element Vector{SnoopCompile.MethodInvalidations}:
...you'll see that it's our deprecated convert methods in ImageCore causing the vast majority of these. So then I dev ImageCore, comment out the include("deprecations.jl"), and try again:
julia> using SnoopCompileCore
julia> invs = @snoopr using ImageQualityIndexes;
[ Info: Precompiling ImageQualityIndexes [2996bd0c-7a13-11e9-2da2-2f5ce47296a9]
julia> using SnoopCompile
julia> length(uinvalidated(invs))
102Much better. (The trees don't show any particular problem with this package, as the triggers are from its dependencies, at least when run in a fresh session.)
Sadly, when I try again, it still doesn't "take." That could be because of one or more of these 102 remaining invalidations, but at this point I suspect another source. So then I go take a look at the implementation and see that you need imfilter for assess_ssim (which I could have suspected from the @snoopi trace, of course). But imfilter is obviously not precompiled, so it might be worth taking a step back and go visit ImageFiltering and see if I can get that working. That would definitely be worth doing, but I don't have time at the moment to tackle ImageFiltering to see if it works (sorry).
Just FYI, some other useful tricks include stuff like this:
a = zeros(C{T}, ntuple(i->1, n))
am = of_eltype(float(T), a)
if isdefined(Base, :bodyfunction)
m = which(_ssim_statistics, (GenericImage, GenericImage, Any))
f = Base.bodyfunction(m)
precompile(f, (Bool, typeof(_ssim_statistics), typeof(am), typeof(am)))
...which let you circumvent some of the complexity of types and just construct instances which you then use to build good precompile directives. In general I find this approach makes my precompile files nicely track changes in implementations. I can't think of a good way to automate this, which is why I generally prefer to do stuff by hand.
ashwani-rathee
I tried to build
precompile.jlwith SnoopCompile for ImageQualityIndexes with the following script:I tried both
@snoopiand@snoopcbut failed to see a boost in the first invocation. What's worse, it recompiles more frequently.The following is what's generated by
@snoopc:@timholy Is there anything here I did incorrectly, or simply there's no precompilation gain doing this in ImageQualityIndexes?