search

JuliaReinforcementLearning / ReinforcementLearning.jl

A reinforcement learning package for Julia
https://juliareinforcementlearning.org
Other
590 stars 112 forks source link

RL Env tests fail with latest OpenSpiel patches #1011

Closed joelreymont closed 9 months ago

joelreymont commented 9 months ago

OpenSpiel tests pass and RL Env builds fine but 

A.L.E: Arcade Learning Environment (version 0.6.0)
[Powered by Stella]
Use -help for help screen.
Warning: couldn't load settings file: ./ale.cfg
[ Info: testing OpenSpiel: tic_tac_toe
random policy with OpenSpielEnv: Error During Test at /Users/joelr/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningBase/src/base.jl:201
  Got exception outside of a @test
  MethodError: no method matching information_state_string(::CxxWrap.StdLib.UniquePtrAllocated{State}, ::Int32)

  Closest candidates are:
    information_state_string(::CxxWrap.StdLib.UniquePtrAllocated{State})
     @ OpenSpiel ~/.julia/packages/OpenSpiel/kjP7Z/src/patch.jl:89
    information_state_string(::Union{State, CxxWrap.CxxWrapCore.CxxRef{<:State}}, ::Integer)
     @ OpenSpiel ~/.julia/packages/CxxWrap/5IZvn/src/CxxWrap.jl:624

  Stacktrace:
    [1] _state(env::OpenSpielEnv{CxxWrap.StdLib.UniquePtrAllocated{State}, CxxWrap.StdLib.SharedPtrAllocated{Game}}, ::InformationSet{String}, player::Int32)
      @ ReinforcementLearningEnvironments ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/src/environments/3rd_party/open_spiel.jl:139
    [2] state(env::OpenSpielEnv{CxxWrap.StdLib.UniquePtrAllocated{State}, CxxWrap.StdLib.SharedPtrAllocated{Game}}, ss::InformationSet{String}, player::Int32)
      @ ReinforcementLearningEnvironments ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/src/environments/3rd_party/open_spiel.jl:135
    [3] state
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningBase/src/interface.jl:490 [inlined]
    [4] state
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningBase/src/interface.jl:489 [inlined]
    [5] macro expansion
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningBase/src/base.jl:214 [inlined]
    [6] macro expansion
      @ ~/.julia/juliaup/julia-1.10.1+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
    [7] test_runnable!(env::OpenSpielEnv{CxxWrap.StdLib.UniquePtrAllocated{State}, CxxWrap.StdLib.SharedPtrAllocated{Game}}, n::Int64; rng::TaskLocalRNG)
      @ ReinforcementLearningBase ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningBase/src/base.jl:202
    [8] test_runnable! (repeats 2 times)
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningBase/src/base.jl:200 [inlined]
    [9] macro expansion
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/environments/3rd_party/open_spiel.jl:9 [inlined]
   [10] macro expansion
      @ ~/.julia/juliaup/julia-1.10.1+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [11] top-level scope
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/environments/3rd_party/open_spiel.jl:2
   [12] include
      @ ./client.jl:489 [inlined]
   [13] macro expansion
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/environments/3rd_party/3rd_party.jl:5 [inlined]
   [14] macro expansion
      @ ~/.julia/juliaup/julia-1.10.1+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [15] top-level scope
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/environments/3rd_party/3rd_party.jl:3
   [16] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [17] top-level scope
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/environments/environments.jl:2
   [18] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [19] macro expansion
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/runtests.jl:20 [inlined]
   [20] macro expansion
      @ ~/.julia/juliaup/julia-1.10.1+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [21] top-level scope
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/runtests.jl:20
   [22] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [23] top-level scope
      @ none:6
   [24] eval
      @ ./boot.jl:385 [inlined]
   [25] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:291
   [26] _start()
      @ Base ./client.jl:552
[ Info: testing OpenSpiel: kuhn_poker
┌ Warning: unexpected player -1, falling back to default state value.
└ @ ReinforcementLearningEnvironments ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/src/environments/3rd_party/open_spiel.jl:127
random policy with OpenSpielEnv: Error During Test at /Users/joelr/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningBase/src/base.jl:201
  Got exception outside of a @test
  MethodError: no method matching information_state_string(::CxxWrap.StdLib.UniquePtrAllocated{State}, ::Int32)

  Closest candidates are:
    information_state_string(::CxxWrap.StdLib.UniquePtrAllocated{State})
     @ OpenSpiel ~/.julia/packages/OpenSpiel/kjP7Z/src/patch.jl:89
    information_state_string(::Union{State, CxxWrap.CxxWrapCore.CxxRef{<:State}}, ::Integer)
     @ OpenSpiel ~/.julia/packages/CxxWrap/5IZvn/src/CxxWrap.jl:624

  Stacktrace:
    [1] _state(env::OpenSpielEnv{CxxWrap.StdLib.UniquePtrAllocated{State}, CxxWrap.StdLib.SharedPtrAllocated{Game}}, ::InformationSet{String}, player::Int32)
      @ ReinforcementLearningEnvironments ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/src/environments/3rd_party/open_spiel.jl:139
    [2] state(env::OpenSpielEnv{CxxWrap.StdLib.UniquePtrAllocated{State}, CxxWrap.StdLib.SharedPtrAllocated{Game}}, ss::InformationSet{String}, player::Int32)
      @ ReinforcementLearningEnvironments ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/src/environments/3rd_party/open_spiel.jl:135
    [3] state
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningBase/src/interface.jl:490 [inlined]
    [4] state
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningBase/src/interface.jl:489 [inlined]
    [5] macro expansion
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningBase/src/base.jl:214 [inlined]
    [6] macro expansion
      @ ~/.julia/juliaup/julia-1.10.1+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
    [7] test_runnable!(env::OpenSpielEnv{CxxWrap.StdLib.UniquePtrAllocated{State}, CxxWrap.StdLib.SharedPtrAllocated{Game}}, n::Int64; rng::TaskLocalRNG)
      @ ReinforcementLearningBase ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningBase/src/base.jl:202
    [8] test_runnable! (repeats 2 times)
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningBase/src/base.jl:200 [inlined]
    [9] macro expansion
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/environments/3rd_party/open_spiel.jl:9 [inlined]
   [10] macro expansion
      @ ~/.julia/juliaup/julia-1.10.1+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [11] top-level scope
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/environments/3rd_party/open_spiel.jl:2
   [12] include
      @ ./client.jl:489 [inlined]
   [13] macro expansion
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/environments/3rd_party/3rd_party.jl:5 [inlined]
   [14] macro expansion
      @ ~/.julia/juliaup/julia-1.10.1+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [15] top-level scope
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/environments/3rd_party/3rd_party.jl:3
   [16] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [17] top-level scope
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/environments/environments.jl:2
   [18] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [19] macro expansion
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/runtests.jl:20 [inlined]
   [20] macro expansion
      @ ~/.julia/juliaup/julia-1.10.1+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [21] top-level scope
      @ ~/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/runtests.jl:20
   [22] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [23] top-level scope
      @ none:6
   [24] eval
      @ ./boot.jl:385 [inlined]
   [25] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:291
   [26] _start()
      @ Base ./client.jl:552
[ Info: testing OpenSpiel: goofspiel(imp_info=True,num_cards=4,points_order=descending)
┌ Info: testing TigerProblemEnv |> ActionTransformedEnv, you need to manually check these traits to make sure they are implemented correctly!
│   NumAgentStyle(env) = SingleAgent()
│   DynamicStyle(env) = Sequential()
│   ActionStyle(env) = MinimalActionSet()
│   InformationStyle(env) = ImperfectInformation()
│   StateStyle(env) = (Observation{Int64}(), InternalState{Int64}())
│   RewardStyle(env) = StepReward()
│   UtilityStyle(env) = GeneralSum()
└   ChanceStyle(env) = Stochastic()
┌ Info: testing TigerProblemEnv |> DefaultStateStyleEnv, you need to manually check these traits to make sure they are implemented correctly!
│   NumAgentStyle(env) = SingleAgent()
│   DynamicStyle(env) = Sequential()
│   ActionStyle(env) = MinimalActionSet()
│   InformationStyle(env) = ImperfectInformation()
│   StateStyle(env) = (Observation{Int64}(), InternalState{Int64}())
│   RewardStyle(env) = StepReward()
│   UtilityStyle(env) = GeneralSum()
└   ChanceStyle(env) = Stochastic()
┌ Info: testing TigerProblemEnv |> MaxTimeoutEnv, you need to manually check these traits to make sure they are implemented correctly!
│   NumAgentStyle(env) = SingleAgent()
│   DynamicStyle(env) = Sequential()
│   ActionStyle(env) = MinimalActionSet()
│   InformationStyle(env) = ImperfectInformation()
│   StateStyle(env) = (Observation{Int64}(), InternalState{Int64}())
│   RewardStyle(env) = StepReward()
│   UtilityStyle(env) = GeneralSum()
└   ChanceStyle(env) = Stochastic()
┌ Info: testing TigerProblemEnv |> RewardTransformedEnv, you need to manually check these traits to make sure they are implemented correctly!
│   NumAgentStyle(env) = SingleAgent()
│   DynamicStyle(env) = Sequential()
│   ActionStyle(env) = MinimalActionSet()
│   InformationStyle(env) = ImperfectInformation()
│   StateStyle(env) = (Observation{Int64}(), InternalState{Int64}())
│   RewardStyle(env) = StepReward()
│   UtilityStyle(env) = GeneralSum()
└   ChanceStyle(env) = Stochastic()
┌ Info: testing TigerProblemEnv |> RewardOverriddenEnv, you need to manually check these traits to make sure they are implemented correctly!
│   NumAgentStyle(env) = SingleAgent()
│   DynamicStyle(env) = Sequential()
│   ActionStyle(env) = MinimalActionSet()
│   InformationStyle(env) = ImperfectInformation()
│   StateStyle(env) = (Observation{Int64}(), InternalState{Int64}())
│   RewardStyle(env) = StepReward()
│   UtilityStyle(env) = GeneralSum()
└   ChanceStyle(env) = Stochastic()
┌ Info: testing CartPoleEnv |> StateCachedEnv, you need to manually check these traits to make sure they are implemented correctly!
│   NumAgentStyle(env) = SingleAgent()
│   DynamicStyle(env) = Sequential()
│   ActionStyle(env) = MinimalActionSet()
│   InformationStyle(env) = ImperfectInformation()
│   StateStyle(env) = Observation{Any}()
│   RewardStyle(env) = StepReward()
│   UtilityStyle(env) = GeneralSum()
└   ChanceStyle(env) = Stochastic()
┌ Info: testing KuhnPokerEnv |> StochasticEnv, you need to manually check these traits to make sure they are implemented correctly!
│   NumAgentStyle(env) = MultiAgent{2}()
│   DynamicStyle(env) = Sequential()
│   ActionStyle(env) = MinimalActionSet()
│   InformationStyle(env) = ImperfectInformation()
│   StateStyle(env) = InformationSet{Tuple{Vararg{Symbol}}}()
│   RewardStyle(env) = TerminalReward()
│   UtilityStyle(env) = ZeroSum()
└   ChanceStyle(env) = Stochastic()
┌ Info: testing RockPaperScissorsEnv, you need to manually check these traits to make sure they are implemented correctly!
│   NumAgentStyle(env) = MultiAgent{2}()
│   DynamicStyle(env) = Simultaneous()
│   ActionStyle(env) = MinimalActionSet()
│   InformationStyle(env) = ImperfectInformation()
│   StateStyle(env) = Observation{Int64}()
│   RewardStyle(env) = TerminalReward()
│   UtilityStyle(env) = ZeroSum()
└   ChanceStyle(env) = Deterministic()
Test Summary:                         |   Pass  Error   Total   Time
ReinforcementLearningEnvironments     | 380291      2  380293  40.2s
  examples                            | 214307         214307   9.2s
  3rd_party                           | 149428      2  149430  29.7s
    acrobot_env                       |   3211           3211   1.4s
    atari                             | 144211         144211  27.2s
    gym envs                          |                  None   0.1s
    OpenSpielEnv                      |   2006      2    2008   0.8s
      random policy with OpenSpielEnv |      1      1       2   0.7s
      random policy with OpenSpielEnv |      5      1       6   0.0s
      random policy with OpenSpielEnv |   2000           2000   0.0s
    snake game env                    |                  None   0.0s
  wrappers                            |  16556          16556   1.2s
ERROR: LoadError: Some tests did not pass: 380291 passed, 0 failed, 2 errored, 0 broken.
in expression starting at /Users/joelr/Work/Julia/ReinforcementLearning.jl/src/ReinforcementLearningEnvironments/test/runtests.jl:19
ERROR: Package ReinforcementLearningEnvironments errored during testing
jeremiahpslewis commented 9 months ago

I'm looking into it