search

JuliaPOMDP / RockSample.jl

Implement the rock sample problem using POMDPs.jl
Other
5 stars 5 forks source link

POMDPs.jl could not find an implementation for DDN Node :sp #8

Closed Wu-Chenyang closed 4 years ago

Wu-Chenyang commented 4 years ago

First, I defined a rollout policy:

move_east = FunctionPolicy() do b
    return RSPos(1,0)
end

Second, I used it to estimate values for POMCPOW:

value_estimator = FORollout(move_east)
solver = POMCPOWSolver(tree_queries=200000, max_time=1.0, criterion=MaxUCB(10), estimate_value=value_estimator, tree_in_info=true)
planner = solve(solver, pomdp)
b0 = initialstate_distribution(pomdp)
a, info = action_info(planner, b0)

Last, I got output as follows.

POMDPs.jl could not find an implementation for DDN Node :sp. Consider the following options:

1) Implement POMDPs.gen(::RockSamplePOMDP{5}, ::RSState{5}, ::Int64, ::AbstractRNG) to return a NamedTuple with key :sp.

Closest candidates are:
  gen(::Union{MDP, POMDP}, ::Any, ::Any, ::Any) at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\gen_impl.jl:82
  gen(!Matched::GenericDDNNode, ::Any...) at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\ddn_struct.jl:165
  gen(!Matched::DistributionDDNNode, ::Any, ::Any...) at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\ddn_struct.jl:117
  ...

2) Implement POMDPs.gen(::DDNNode{:sp}, ::RockSamplePOMDP{5}, ::Any, ::Any, ::Any, ::AbstractRNG).

Closest candidates are:
  gen(::DDNNode{x}, ::Any, ::Any...) where x at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\gen_impl.jl:67
  gen(!Matched::GenericDDNNode, ::Any...) at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\ddn_struct.jl:165
  gen(!Matched::DistributionDDNNode, ::Any, ::Any...) at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\ddn_struct.jl:117
  ...

3) Implement transition(::RockSamplePOMDP{5}, ::Any, ::Any, ::Any).

Closest candidates are:
  transition(::Any, ::Any, ::Any, ::Any) at C:\Users\hpghj\OneDrive\Workspace\julia\AutomobilePOMDP\RockSample.jl\src\transition.jl:28
  transition(::RockSamplePOMDP{K}, ::RSState{K}, ::Int64) where K at C:\Users\hpghj\OneDrive\Workspace\julia\AutomobilePOMDP\RockSample.jl\src\transition.jl:2
  transition(::Any, ::Any, ::Any) at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\errors.jl:107
  ...

This error message uses heuristics to make recommendations for POMDPs.jl problem implementers. If it was misleading or you believe there is an inconsistency, please file an issue: https://github.com/JuliaPOMDP/POMDPs.jl/issues/new

Stacktrace:
 [1] distribution_impl_error(::Symbol, ::Function, ::Type{T} where T, ::Array{DataType,1}) at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\errors.jl:77
 [2] transition(::RockSamplePOMDP{5}, ::RSState{5}, ::StaticArrays.SArray{Tuple{2},Int64,1,2}) at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\errors.jl:107
 [3] macro expansion at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\ddn_struct.jl:119 [inlined]
 [4] gen(::DistributionDDNNode{typeof(transition)}, ::RockSamplePOMDP{5}, ::RSState{5}, ::StaticArrays.SArray{Tuple{2},Int64,1,2}, ::LBDESPOT.MemorizingRNG{LBDESPOT.MemorizingSource{MersenneTwister}}) at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\ddn_struct.jl:117
 [5] macro expansion at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\gen_impl.jl:78 [inlined]
 [6] gen(::DDNNode{:sp}, ::RockSamplePOMDP{5}, ::RSState{5}, ::StaticArrays.SArray{Tuple{2},Int64,1,2}, ::LBDESPOT.MemorizingRNG{LBDESPOT.MemorizingSource{MersenneTwister}}) at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\gen_impl.jl:67
 [7] macro expansion at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\gen_impl.jl:46 [inlined]
 [8] macro expansion at .\none:0 [inlined]
 [9] genout(::DDNOut{(:sp, :o, :r)}, ::DDNStructure{NamedTuple{(:s, :a, :sp, :o, :r),Tuple{InputDDNNode,InputDDNNode,DistributionDDNNode{typeof(transition)},DistributionDDNNode{typeof(observation)},FunctionDDNNode{typeof(reward)}}},NamedTuple{(:s, :a, :sp, :o, :r),Tuple{Tuple{},Tuple{},Tuple{DDNNode{:s},DDNNode{:a}},Tuple{DDNNode{:s},DDNNode{:a},DDNNode{:sp}},Tuple{DDNNode{:s},DDNNode{:a},DDNNode{:sp},DDNNode{:o}}}}}, ::RockSamplePOMDP{5}, ::RSState{5}, ::StaticArrays.SArray{Tuple{2},Int64,1,2}, ::LBDESPOT.MemorizingRNG{LBDESPOT.MemorizingSource{MersenneTwister}}) at .\none:0
 [10] macro expansion at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\gen_impl.jl:19 [inlined]
 [11] gen(::DDNOut{(:sp, :o, :r)}, ::RockSamplePOMDP{5}, ::RSState{5}, ::StaticArrays.SArray{Tuple{2},Int64,1,2}, ::LBDESPOT.MemorizingRNG{LBDESPOT.MemorizingSource{MersenneTwister}}) at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\gen_impl.jl:4
 [12] (::LBDESPOT.var"#8#9")(::RockSamplePOMDP{5}, ::Vararg{Any,N} where N) at C:\Users\hpghj\.julia\packages\POMDPs\0djkf\src\generative.jl:187
 [13] branching_sim(::RockSamplePOMDP{5}, ::FunctionPolicy{var"#3#4"}, ::LBDESPOT.ScenarioBelief{RSState{5},SparseCat{Array{RSState{5},1},Array{Float64,1}},LBDESPOT.MemorizingSource{MersenneTwister}}, ::Int64, ::LBDESPOT.var"#23#25") at C:\Users\hpghj\OneDrive\Workspace\julia\AutomobilePOMDP\LB-DESPOT\src\default_policy_sim.jl:17
 [14] lbound at C:\Users\hpghj\OneDrive\Workspace\julia\AutomobilePOMDP\LB-DESPOT\src\bounds.jl:116 [inlined]
 [15] bounds at C:\Users\hpghj\OneDrive\Workspace\julia\AutomobilePOMDP\LB-DESPOT\src\bounds.jl:50 [inlined]
 [16] LBDESPOT.LB_DESPOT(::LB_DESPOTPlanner{RockSamplePOMDP{5},IndependentBounds{DefaultPolicyLB{FunctionPolicy{var"#3#4"},Int64,LBDESPOT.var"#23#25"},FullyObservableValueUB{AlphaVectorPolicy{RockSamplePOMDP{5},Int64}}},LBDESPOT.MemorizingSource{MersenneTwister},MersenneTwister}, ::SparseCat{Array{RSState{5},1},Array{Float64,1}}) at C:\Users\hpghj\OneDrive\Workspace\julia\AutomobilePOMDP\LB-DESPOT\src\tree.jl:31
 [17] build_despot at C:\Users\hpghj\OneDrive\Workspace\julia\AutomobilePOMDP\LB-DESPOT\src\planner.jl:2 [inlined]
 [18] action_info(::LB_DESPOTPlanner{RockSamplePOMDP{5},IndependentBounds{DefaultPolicyLB{FunctionPolicy{var"#3#4"},Int64,LBDESPOT.var"#23#25"},FullyObservableValueUB{AlphaVectorPolicy{RockSamplePOMDP{5},Int64}}},LBDESPOT.MemorizingSource{MersenneTwister},MersenneTwister}, ::SparseCat{Array{RSState{5},1},Array{Float64,1}}) at C:\Users\hpghj\OneDrive\Workspace\julia\AutomobilePOMDP\LB-DESPOT\src\pomdps_glue.jl:8
 [19] top-level scope at In[3]:7

Here shows the version of my packages:

  [8bb6e9a1] BeliefUpdaters v0.2.0
  [336ed68f] CSV v0.7.7
  [159f3aea] Cairo v1.0.5
  [a93c6f00] DataFrames v0.21.6
  [186bb1d3] Fontconfig v0.4.0
  [4c53ee00] POMCPOW v0.3.1
  [08074719] POMDPModelTools v0.2.4
  [182e52fb] POMDPPolicies v0.3.3
  [e0d0a172] POMDPSimulators v0.3.1
  [a93abf59] POMDPs v0.8.4
  [8ba89e20] Distributed
  [de0858da] Printf
  [9a3f8284] Random
  [10745b16] Statistics

Besides, for compatibility, I reset the git HEAD to the commit before "update to pomdps 0.9" with command git reset --hard 21600e68ae0f633b6ad7ba360429dc4299654d7a How can I fix this issue?

MaximeBouton commented 4 years ago

Hi, It seems like it is passing a StaticArrays.SArray{Tuple{2},Int64,1,2} instead of Int64 for the action when calling transition and causes a method error. I have to admit the error message is not super clear.

I think it is because the move_east policy returns the wrong object, it returns a RSPos instead of an action. If you want a policy that always return the action move east then it should be something like that.

move_east = FunctionPolicy() do b
    return RockSample.BASIC_ACTIONS_DICT[:east] # return an action (::Int64)
end
Wu-Chenyang commented 4 years ago

Ah! Thanks for quick answer. That solved my problem. 👍