JuliaReinforcementLearning / CommonRLInterface.jl

A minimal reinforcement learning environment interface with additional opt-in features.
MIT License
45 stars 1 forks source link

`Optional` and `AutomaticDefault` submodules #44

Closed zsunberg closed 1 year ago

zsunberg commented 3 years ago

I propose putting all optional functions, provided, and @provide in an Optional submodule.

I think we should also have a Default submodule that implements defaults wherever possible.

For example:

Default.clone(env::AbstractEnv) = deepcopy(env)

or

function Default.valid_actions(env::AbstractEnv)
    if provided(Optional.valid_actions, env)
         return Optional.valid_actions(env)
    elseif provided(Optional.valid_action_mask, env)
        return actions(env)[Optional.valid_action_mask(env)]
    else
        return actions(env)
    end
end

If a solver/agent wants to know whether an environment explicitly provides some optional behavior, they could use provided, but if they want to use it for any environment and automatically fall back to the default, they can use, and it will work for any env.

using CommonRLInterface.Default: valid_actions

valid_actions(env) # will work for any environment
Optional.valid_actions(env) # will throw a MethodError if the environment does not implement `valid_actions`
zsunberg commented 3 years ago

There may be a better name than Default like AutomaticDefault

zsunberg commented 3 years ago

The more I think about this, the more I think this is a good idea... I am going to try to implement it and see how it comes out.