search

JuliaML / Reinforce.jl

Abstractions, algorithms, and utilities for reinforcement learning in Julia
Other
201 stars 35 forks source link

Mountain Car Example is not working #33

Closed mbu93 closed 5 years ago

mbu93 commented 5 years ago

Expectation

With the latest release of Reinforce.jl and a higher julia version than julia 0.6 the Mountain Car Example should work and produce a plot like element

Actual Behaviour

Using the current Julia version will cause the old version of src files (including linspace instead of range) to be installed. Even for installing the git version with Pkg.clone([url]) for most current files the example seems to be broken. Running it from within julia will throw the follwing message:

MethodError: no method matching sin(::StepRangeLen{Float64,Base.TwicePrecision{Float64},Base.TwicePrecision{Float64}})
Closest candidates are:
  sin(!Matched::BigFloat) at mpfr.jl:683
  sin(!Matched::Missing) at math.jl:1070
  sin(!Matched::Complex{Float16}) at math.jl:1019
  ...

Stacktrace:
 [1] height(::StepRangeLen{Float64,Base.TwicePrecision{Float64},Base.TwicePrecision{Float64}}) at /home/mbauer/.julia/dev/Reinforce/src/envs/mountain_car.jl:84
 [2] macro expansion at /home/mbauer/.julia/dev/Reinforce/src/envs/mountain_car.jl:101 [inlined]
 [3] macro expansion at /home/mbauer/.julia/packages/RecipesBase/Uz5AO/src/RecipesBase.jl:312 [inlined]
 [4] macro expansion at /home/mbauer/.julia/dev/Reinforce/src/envs/mountain_car.jl:99 [inlined]
 [5] apply_recipe(::Dict{Symbol,Any}, ::MountainCar) at /home/mbauer/.julia/packages/RecipesBase/Uz5AO/src/RecipesBase.jl:275
 [6] _process_userrecipes(::Plots.Plot{Plots.GRBackend}, ::Dict{Symbol,Any}, ::Tuple{MountainCar}) at /home/mbauer/.julia/packages/Plots/rmogG/src/pipeline.jl:83
 [7] macro expansion at ./logging.jl:305 [inlined]
 [8] _plot!(::Plots.Plot{Plots.GRBackend}, ::Dict{Symbol,Any}, ::Tuple{MountainCar}) at /home/mbauer/.julia/packages/Plots/rmogG/src/plot.jl:171
 [9] #plot#132(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::MountainCar) at /home/mbauer/.julia/packages/Plots/rmogG/src/plot.jl:57
 [10] plot at /home/mbauer/.julia/packages/Plots/rmogG/src/plot.jl:51 [inlined]
 [11] episode!(::MountainCar, ::BasicCarPolicy) at ./In[3]:12
 [12] top-level scope at In[3]:16
# Steps to Reproduce the Problem

This is due to the shortcut defined in line 84 of __mountain_car.jl__

height(xs) = sin(3*xs)*0.45+0.55

since this operation will not work for the vector in line 100:

xs = range(min_position, max_position, length=100)
ys = height(xs)

Anyways, eliminating this error using a vectorized function call and LinRange instead of range like this

xs = LinRange(min_position, max_position, 100)
ys = height.(xs)

will cause compilation to fail with the following message:

MethodError: no method matching +(::Array{Float64,1}, ::Float64)
Closest candidates are:
  +(::Any, ::Any, !Matched::Any, !Matched::Any...) at operators.jl:502
  +(!Matched::Bool, ::T<:AbstractFloat) where T<:AbstractFloat at bool.jl:112
  +(!Matched::Float64, ::Float64) at float.jl:395
  ...

Stacktrace:
 [1] apply_recipe(::Dict{Symbol,Any}, ::MountainCar) at /home/mbauer/.julia/packages/RecipesBase/Uz5AO/src/RecipesBase.jl:314
 [2] _process_userrecipes(::Plots.Plot{Plots.GRBackend}, ::Dict{Symbol,Any}, ::Tuple{MountainCar}) at /home/mbauer/.julia/packages/Plots/rmogG/src/pipeline.jl:83
 [3] macro expansion at ./logging.jl:305 [inlined]
 [4] _plot!(::Plots.Plot{Plots.GRBackend}, ::Dict{Symbol,Any}, ::Tuple{MountainCar}) at /home/mbauer/.julia/packages/Plots/rmogG/src/plot.jl:171
 [5] #plot#132(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::MountainCar) at /home/mbauer/.julia/packages/Plots/rmogG/src/plot.jl:57
 [6] plot at /home/mbauer/.julia/packages/Plots/rmogG/src/plot.jl:51 [inlined]
 [7] episode!(::MountainCar, ::BasicCarPolicy) at ./In[2]:12
 [8] top-level scope at In[2]:17

Obviously now there is some line where a none elementwise addition of the now array like ys will cause an error. Since I am new to Julia that*s basically all I coudl figure out. A little help (or a fix) would be really much appreciated!

Steps to reproduce

Specifications

The following setup was used

iblislin commented 5 years ago

Hi @mbu93, thanks for this great report. Could you try out #34? I currently do not have X11 available, and not sure about plotting is fixed or not.

mbu93 commented 5 years ago

Hi @iblis17!

The example works with fix #34! Thank you for your quick reply.