DictSrv with error resiliency?

[tbenst]

Suppose I'm using the distributed dict example from tutorial, and I do something that throws an error. now, my Actor is permanently broken. It would be great to have more resiliency. I'm reading the docs on Supervisors, but it's not completely clear to me how to implement this, although it seems possible.

In particular, I'm not sure how to supervise Actors that are spawned inside of a function--unless I do something messy like append each spawn to a global Array..? Feels like an anti-pattern but maybe that's the best way?

What happens:

julia> include("./mydict.jl");
julia> using .MyDict;
julia> d = dictsrv(Dict{Int,Int}());
julia> d[2] = 1
1

julia> d()
Dict{Int64, Int64} with 1 entry:
  2 => 1

julia> d[:a] = 1 # oops! this isn't valid since dict has index of Int
1

julia> d() # and now, our DictSrv is permanently broken
ERROR: TaskFailedException
Stacktrace:
 [1] check_channel_state
   @ ./channels.jl:170 [inlined]
 [2] _send!(chn::Channel{Any}, msg::Actors.Call)
   @ Actors ~/.julia/packages/Actors/skg0T/src/com.jl:7
 [3] send
   @ ~/.julia/packages/Actors/skg0T/src/com.jl:42 [inlined]
 [4] request(lk::Actors.Link{Channel{Any}}, msg::Actors.Call; full::Bool, timeout::Float64)
   @ Actors ~/.julia/packages/Actors/skg0T/src/com.jl:154
 [5] #request#29
   @ ~/.julia/packages/Actors/skg0T/src/com.jl:162 [inlined]
 [6] #call#157
   @ ~/.julia/packages/Actors/skg0T/src/api.jl:43 [inlined]
 [7] call
   @ ~/.julia/packages/Actors/skg0T/src/api.jl:43 [inlined]
 [8] (::DictSrv{Actors.Link{Channel{Any}}})()
   @ Main.MyDict /tmp/mydict.jl:11
 [9] top-level scope
   @ REPL[7]:1

    nested task error: MethodError: Cannot `convert` an object of type Symbol to an object of type Int64
    Closest candidates are:
      convert(::Type{T}, ::Ptr) where T<:Integer at pointer.jl:23
      convert(::Type{T}, ::T) where T<:Number at number.jl:6
      convert(::Type{T}, ::Number) where T<:Number at number.jl:7
      ...
    Stacktrace:
     [1] setindex!(h::Dict{Int64, Int64}, v0::Int64, key0::Symbol)
       @ Base ./dict.jl:374
     [2] ds(::Dict{Int64, Int64}, ::Function, ::Int64, ::Vararg{Any, N} where N)
       @ Main.MyDict /tmp/mydict.jl:18
     [3] (::Actors.var"#4#6"{Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, typeof(Main.MyDict.ds), Tuple{Dict{Int64, Int64}}})(::Function, ::Vararg{Any, N} where N)
       @ Actors ~/.julia/packages/Actors/skg0T/src/types.jl:42
     [4] (::Actors.Bhv{Actors.var"#4#6"{Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, typeof(Main.MyDict.ds), Tuple{Dict{Int64, Int64}}}})(::Function, ::Vararg{Any, N} where N)
       @ Actors ~/.julia/packages/Actors/skg0T/src/types.jl:46
     [5] onmessage(A::Actors._ACT, msg::Actors.Call)
       @ Actors ~/.julia/packages/Actors/skg0T/src/protocol.jl:20
     [6] onmessage(A::Actors._ACT, mode::Val{:default}, msg::Actors.Call)
       @ Actors ~/.julia/packages/Actors/skg0T/src/actor.jl:53
     [7] _act(ch::Channel{Any})
       @ Actors ~/.julia/packages/Actors/skg0T/src/actor.jl:67
     [8] (::Actors.var"#148#152")()
       @ Actors ~/.julia/packages/Actors/skg0T/src/actor.jl:123

julia> d[3] = 1 # still broken
ERROR: TaskFailedException
Stacktrace:
 [1] check_channel_state
   @ ./channels.jl:170 [inlined]
 [2] _send!(chn::Channel{Any}, msg::Actors.Call)
   @ Actors ~/.julia/packages/Actors/skg0T/src/com.jl:7
 [3] send
   @ ~/.julia/packages/Actors/skg0T/src/com.jl:42 [inlined]
 [4] request(lk::Actors.Link{Channel{Any}}, msg::Actors.Call; full::Bool, timeout::Float64)
   @ Actors ~/.julia/packages/Actors/skg0T/src/com.jl:154
 [5] #request#29
   @ ~/.julia/packages/Actors/skg0T/src/com.jl:162 [inlined]
 [6] #call#157
   @ ~/.julia/packages/Actors/skg0T/src/api.jl:43 [inlined]
 [7] call
   @ ~/.julia/packages/Actors/skg0T/src/api.jl:43 [inlined]
 [8] setindex!(d::DictSrv{Actors.Link{Channel{Any}}}, value::Int64, key::Int64)
   @ Main.MyDict /tmp/mydict.jl:15
 [9] top-level scope
   @ REPL[8]:1

    nested task error: MethodError: Cannot `convert` an object of type Symbol to an object of type Int64
    Closest candidates are:
      convert(::Type{T}, ::Ptr) where T<:Integer at pointer.jl:23
      convert(::Type{T}, ::T) where T<:Number at number.jl:6
      convert(::Type{T}, ::Number) where T<:Number at number.jl:7
      ...
    Stacktrace:
     [1] setindex!(h::Dict{Int64, Int64}, v0::Int64, key0::Symbol)
       @ Base ./dict.jl:374
     [2] ds(::Dict{Int64, Int64}, ::Function, ::Int64, ::Vararg{Any, N} where N)
       @ Main.MyDict /tmp/mydict.jl:18
     [3] (::Actors.var"#4#6"{Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, typeof(Main.MyDict.ds), Tuple{Dict{Int64, Int64}}})(::Function, ::Vararg{Any, N} where N)
       @ Actors ~/.julia/packages/Actors/skg0T/src/types.jl:42
     [4] (::Actors.Bhv{Actors.var"#4#6"{Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, typeof(Main.MyDict.ds), Tuple{Dict{Int64, Int64}}}})(::Function, ::Vararg{Any, N} where N)
       @ Actors ~/.julia/packages/Actors/skg0T/src/types.jl:46
     [5] onmessage(A::Actors._ACT, msg::Actors.Call)
       @ Actors ~/.julia/packages/Actors/skg0T/src/protocol.jl:20
     [6] onmessage(A::Actors._ACT, mode::Val{:default}, msg::Actors.Call)
       @ Actors ~/.julia/packages/Actors/skg0T/src/actor.jl:53
     [7] _act(ch::Channel{Any})
       @ Actors ~/.julia/packages/Actors/skg0T/src/actor.jl:67
     [8] (::Actors.var"#148#152")()
       @ Actors ~/.julia/packages/Actors/skg0T/src/actor.jl:123

What I want:

julia> include("./mydict.jl");
julia> using .MyDict;
julia> d = dictsrv(Dict{Int,Int}());
julia> d[2] = 1
1

julia> d()
Dict{Int64, Int64} with 1 entry:
  2 => 1

julia> d[:a] = 1 # oops! this isn't valid since dict has index of Int
1
[... show error sometime later]
julia> d[3] = 1 # not broken!
1

julia> d()
Dict{Int64, Int64} with 1 entry:
  2 => 1,
  3 => 1

[tbenst]

Ok, I think I came up with a relatively simple solution by modifying dictsrv:

function dictsrv(d::Dict; remote=false)
    sv = supervisor()
    dsrv = DictSrv(spawn(ds, d; remote))
    exec(dsrv.lk, Actors.supervise, sv);
    dsrv
end

I suppose this is a bit inefficient, as I spawn one supervisor per actor, when it might be better to have one master supervisor? Would sv as a global variable be preferred?