search

JuliaReinforcementLearning / ReinforcementLearningAnIntroduction.jl

Julia code for the book Reinforcement Learning An Introduction
https://juliareinforcementlearning.org/ReinforcementLearningAnIntroduction.jl/
MIT License
309 stars 58 forks source link

method not implemented in Chapter01 notebook #64

Closed kir0ul closed 3 years ago

kir0ul commented 3 years ago

At the following cell in Chapter01_Tic_Tac_Toe.jl notebook:

run(policies, E, StopAfterEpisode(100_000))

I get the following error:

method not implemented

    error(::String)@error.jl:33
    action_space(::ReinforcementLearningEnvironments.TicTacToeEnv, ::ReinforcementLearningEnvironments.Nought)@none:0
    var"#action_space#22"(::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, ::typeof(ReinforcementLearningBase.action_space), ::ReinforcementLearningEnvironments.DefaultStateStyleEnv{ReinforcementLearningBase.Observation{Int64}(), ReinforcementLearningEnvironments.TicTacToeEnv}, ::ReinforcementLearningEnvironments.Nought)@wrappers.jl:17
    action_space(::ReinforcementLearningEnvironments.DefaultStateStyleEnv{ReinforcementLearningBase.Observation{Int64}(), ReinforcementLearningEnvironments.TicTacToeEnv}, ::ReinforcementLearningEnvironments.Nought)@wrappers.jl:17
    update!(::ReinforcementLearningCore.VectorSARTTrajectory{NamedTuple{(:state, :action, :reward, :terminal), Tuple{Vector{Int64}, Vector{Union{ReinforcementLearningCore.NoOp, Int64}}, Vector{Int64}, Vector{Bool}}}}, ::ReinforcementLearningCore.NamedPolicy{ReinforcementLearningCore.VBasedPolicy{ReinforcementLearningZoo.MonteCarloLearner{ReinforcementLearningCore.TabularVApproximator{Vector{Float64}, Flux.Optimise.InvDecay}, ReinforcementLearningZoo.FirstVisit, ReinforcementLearningZoo.NoSampling}, typeof(Main.workspace2.select_action)}, ReinforcementLearningEnvironments.Nought}, ::ReinforcementLearningEnvironments.DefaultStateStyleEnv{ReinforcementLearningBase.Observation{Int64}(), ReinforcementLearningEnvironments.TicTacToeEnv}, ::ReinforcementLearningCore.PostEpisodeStage)@agent.jl:142
    (::ReinforcementLearningCore.Agent{ReinforcementLearningCore.NamedPolicy{ReinforcementLearningCore.VBasedPolicy{ReinforcementLearningZoo.MonteCarloLearner{ReinforcementLearningCore.TabularVApproximator{Vector{Float64}, Flux.Optimise.InvDecay}, ReinforcementLearningZoo.FirstVisit, ReinforcementLearningZoo.NoSampling}, typeof(Main.workspace2.select_action)}, ReinforcementLearningEnvironments.Nought}, ReinforcementLearningCore.VectorSARTTrajectory{NamedTuple{(:state, :action, :reward, :terminal), Tuple{Vector{Int64}, Vector{Union{ReinforcementLearningCore.NoOp, Int64}}, Vector{Int64}, Vector{Bool}}}}})(::ReinforcementLearningCore.PostEpisodeStage, ::ReinforcementLearningEnvironments.DefaultStateStyleEnv{ReinforcementLearningBase.Observation{Int64}(), ReinforcementLearningEnvironments.TicTacToeEnv})@agent.jl:68
    (::ReinforcementLearningCore.MultiAgentManager)(::ReinforcementLearningCore.PostEpisodeStage, ::ReinforcementLearningEnvironments.DefaultStateStyleEnv{ReinforcementLearningBase.Observation{Int64}(), ReinforcementLearningEnvironments.TicTacToeEnv})@multi_agent.jl:38
    _run(::ReinforcementLearningCore.MultiAgentManager, ::ReinforcementLearningEnvironments.DefaultStateStyleEnv{ReinforcementLearningBase.Observation{Int64}(), ReinforcementLearningEnvironments.TicTacToeEnv}, ::ReinforcementLearningCore.StopAfterEpisode{ProgressMeter.Progress}, ::ReinforcementLearningCore.EmptyHook)@run.jl:44
    run@run.jl:10[inlined]
    run@run.jl:9[inlined]
    top-level scope@Local: 1[inlined]

I believe the notebook was working correctly last time I updated it in https://github.com/JuliaReinforcementLearning/ReinforcementLearningAnIntroduction.jl/commit/3ded6eb4d71c5883e7d9ec734d731cc0246c7284, so not sure what changed since then. Any pointers how to fix this?

findmyway commented 3 years ago

I happen to have some time in the next two days. I'll take a look into it soon.

By the way, I'd be glad to hear your feedback on how to improve this project further 😄 (you can reach me on slack anytime,or we can schedule a quick online meeting)

kir0ul commented 3 years ago

By the way, I'd be glad to hear your feedback on how to improve this project further (you can reach me on slack anytime,or we can schedule a quick online meeting)

Do you mean the Julia Slack? I never joined it but maybe I should have? In case it's useful I'm on the Julia Zulip though.

findmyway commented 3 years ago

I believe the notebook was working correctly last time I updated it in 3ded6eb, so not sure what changed since then. Any pointers how to fix this?

It seems the latest Pluto will download the latest version of dependencies and do not respect the Project.toml and Manifest.toml file in the current directory by default now.

I fixed this bug in another PR.

I'll go over all the notebooks and update them.