JuliaML / Reinforce.jl

Abstractions, algorithms, and utilities for reinforcement learning in Julia
Other
201 stars 35 forks source link

Pendulum: make PendulumState as subtype of AbstractArray #49

Closed lilanger closed 3 years ago

lilanger commented 3 years ago
iblislin commented 3 years ago

changed state from PendulumState type to array (easier to handle in advanced algorithms)

How about keep PendulumState presented but just subtyped it from Vector{Float}, do this way help you handle in your algo? (Or solution 2: wirte some convert function for it.)

lilanger commented 3 years ago

It would be essential that env.state returns an Array instead of a PendulumState. I don't know how to do the subtyping though, I tried something like this but this is not valid:


mutable struct PendulumState{T<:Float64} <: Vector{Float64}
  θ::T
  θvel::T
  PendulumState(θ, θvel) = Float64[θ, θvel]
end

mutable struct Pendulum <: AbstractEnvironment
  state::Vector{Float64}
  reward::Float64
  a::Float64 # last action for rendering
  steps::Int
  maxsteps::Int
end
iblislin commented 3 years ago

ah, I guess the AbstractVector is the legit one.

lilanger commented 3 years ago

Nice, yes this seems to work.

iblislin commented 3 years ago

could you test the new patch?