UFRN-URNAI / urnai-tools

A modular Deep Reinforcement Learning library that supports multiple environments, made with Python 3.6.
Apache License 2.0
5 stars 8 forks source link

Base class for the state representation #72

Closed alvarofpp closed 8 months ago

alvarofpp commented 11 months ago

We need a class that lets us know the agent's current state. It needs to have the update method, which will update the agent's state according to the script developed.

Suggestions:

RickFqt commented 8 months ago

About this state representation class, does it have a similar idea as StateBuilder? Should this update method be something similar to build_state from there?

alvarofpp commented 8 months ago

I believe it is. And I think it's worth reflecting on the methods we have and whether they are really necessary, as well as reflecting on the addition of new methods.

Example:

RickFqt commented 8 months ago
  • If the class is the representation of a State, I believe that build_state is redundant and could just be called build.

In that case, I believe both build and update have the same purpose: given an Observation, create a new State from it. But since this class is the representation of a State, should the class have a @property to store the state ( a list of features extracted from the observation ), or should the method build / update just return it?

alvarofpp commented 8 months ago

should the class have a @property to store the state ( a list of features extracted from the observation ), or should the method build / update just return it?

Both. Imagine that you want to treat the current state twice. If you use build/update, you will execute the function twice to get the same result. To do this, I believe that build/update should save the @property and then return it. That way, you can access the current state as many times as you like, and you only need to run build/update once.

RickFqt commented 8 months ago

I just added my implementation here, but I'm not sure if I used the @property correctly.

alvarofpp commented 8 months ago

Open a PR as a draft, it's better to keep track of your code. It seems that everything is fine with your code, when you develop the tests we'll be able to evaluate it better.