Tagging causing inconsistent behavior

jarredbarber commented 5 years ago

The following code should produce the same output twice. However, when tagging is enabled, wildly different code branches seem to be taken, and the tagging version fails with a MethodError

Code:

# Minimialist example of tagging bug
using Cassette
using Cassette: tag, untag, @context, metadatatype, metadata, hasmetadata, enabletagging, overdub, recurse

@context TagCtx

Cassette.metadatatype(ctx::Type{<:TagCtx}, ::Type) = Type
Cassette.metadatatype(ctx::Type{<:TagCtx}, ::DataType) = Type

function Cassette.overdub(ctx::TagCtx, fn, args...)
    Core.println("fn: $fn")
    val = recurse(ctx, fn, args...)
    hasmetadata(val, ctx) ? val : tag(val, ctx, typeof(val))
end

@context IdentityCtx
function Cassette.overdub(ctx::IdentityCtx, fn, args...)
    Core.println("fn: $fn")
    recurse(ctx, fn, args...)
end

function fn()
    Main.println("help")
end

println("Identity ctx:")
overdub(IdentityCtx(), fn)
println("Tagging ctx:")
overdub(enabletagging(TagCtx(), fn), fn)

When IdentityCtx runs (first 20 lines):

Identity ctx:
fn: fn
fn: println
fn: typeassert
fn: tuple
fn: println
fn: tuple
fn: tuple
fn: print
fn: lock
fn: lock
fn: current_task
fn: ==
fn: ===
fn: ===
fn: setproperty!
fn: typeof
fn: fieldtype
fn: convert
fn: setfield!

When the TaggingCtx runs (first 20 lines)j:

Tagging ctx:
fn: fn
fn: println
fn: iterate
fn: iterate
fn: <=
fn: length
fn: nfields
fn: <=
fn: getindex
fn: +
fn: iterate
fn: <=
fn: length
fn: nfields
fn: <=
fn: iterate
fn: iterate
fn: <=
fn: length

The IdentityCtx case runs to the end, but the tagging one fails after some number of lines, with a very deep stack trace (43 entries):

fn: notify                                                                                                         
fn: Base.#notify#442                                                                                               
fn: notify                                                                                                         
fn: getproperty                                                                                                    
fn: length                                                                                                         
fn: getproperty                                                                                                    
fn: iterate                                                                                                        
fn: iterate                                                                                                        
fn: rem                                                                                                            
fn: -                                                                                                              
fn: Base.promote_typeof                                                                                            
fn: typeof                                                                                                         
fn: Base.promote_typeof                                                                                            
fn: typeof                                                                                                         
fn: promote_type                                                                                                   
fn: rem                                                                                                            
ERROR: LoadError: MethodError: no method matching rem(::UInt64, ::TypeVar)                                         
Closest candidates are:                                                                                            
  rem(::Any, ::Any, !Matched::RoundingMode{:ToZero}) at math.jl:739                                                
  rem(::Any, ::Any, !Matched::RoundingMode{:Down}) at math.jl:740                                                  
  rem(::Any, ::Any, !Matched::RoundingMode{:Up}) at math.jl:741                                                    
  ...                                                                                                              
Stacktrace:                                                                                                        
 [1] call at /home/jarred/.julia/dev/Cassette/src/context.jl:448 [inlined]                                         
 [2] fallback at /home/jarred/.julia/dev/Cassette/src/context.jl:445 [inlined]                                     
 [3] recurse(::Cassette.Context{nametype(TagCtx),Nothing,Cassette.Tag{nametype(TagCtx),0x33c08e1374a2590a,Nothing},g
etfield(Cassette, Symbol("##PassType#365")),IdDict{Module,Dict{Symbol,Cassette.BindingMeta}},Nothing}, ::typeof(rem)
, ::Cassette.Tagged{Cassette.Tag{nametype(TagCtx),0x33c08e1374a2590a,Nothing},UInt64,Type,Cassette.NoMetaMeta,Casset
te.Context{nametype(TagCtx),Nothing,Cassette.Tag{nametype(TagCtx),0x33c08e1374a2590a,Nothing},getfield(Cassette, Sym
bol("##PassType#365")),IdDict{Module,Dict{Symbol,Cassette.BindingMeta}},Nothing}}, ::Cassette.Tagged{Cassette.Tag{na
metype(TagCtx),0x33c08e1374a2590a,Nothing},TypeVar,Type,NamedTuple{(:name, :lb, :ub),Tuple{Cassette.Mutable{Cassette
.Meta{Type,Cassette.NoMetaMeta}},Cassette.Mutable{Cassette.Meta},Cassette.Mutable{Cassette.Meta}}},Cassette.Context{
nametype(TagCtx),Nothing,Cassette.Tag{nametype(TagCtx),0x33c08e1374a2590a,Nothing},getfield(Cassette, Symbol("##Pass
Type#365")),IdDict{Module,Dict{Symbol,Cassette.BindingMeta}},Nothing}}) at /home/jarred/.julia/dev/Cassette/src/over
dub.jl:481

femtomc commented 5 years ago

Note that Core.println actually throws a different error:

ArgumentError: nothing should not be printed; use show, repr, or custom output instead.

I still don't really understand the difference between Main.println and Core.println but this might be related to the async IO comments on the Julia slack.

vchuravy commented 5 years ago

Notes from my exploration today. You shouldn't need to untag yourself. recurse should to that, but we seem to put something into a tuple instead of splatting it.

femtomc commented 5 years ago

Looks like the ntuple-ing on L448 in context.jl:

@inline call(context::ContextTagged, f, args...) = untag(f, context)(ntuple(i -> untag(args[i], context), Val(nfields(args)))...)

Will try a few things if I have time today.

jarredbarber commented 5 years ago

@vchuravy I edited the example to not manually untag things.

JuliaLabs / Cassette.jl

Tagging causing inconsistent behavior #133