cswinter
commented
2 years ago We split out the IDS into entity types... but why don't we just put the IDS along with the entities in the features? Having the same place for both things is more coherent and much easier to debug e.g:
I think we could do this, I actually started out combining all of "actions", "ids", and "features" into a single EntityObs so we have a single entities: Mapping[EntityType, EntityObs], but then changed back to the previous scheme since it made it harder to figure out all the algorithmic changes. I think it probably makes sense to still keep actions separate, since actions feel like a top-level object and often don't make sense to split across entity type, but joining ids and entities seems good. Maybe we could also have a constructor/function on Environment which you pass just the lists of ids and features, and it does the whole np.array(... thing and validates that the number of features match and the number of ids match the number of entities.
Bam4d
This is a large refactor of the
EnvironmentAPI. It includes a new tutorial describing how to implement anEnvironmentThe main change is the wayEnityIDs are handled:Observation.idsis now aMapping[EntityType, Sequence[EntityID]]rather than just aSequence[EntityID]to make the correspondence between entities and IDs more explicit.Observation.idsfor all entity types, only if they are referenced by an action.*ActionMaskclasses now have two ways of specifying the actors:actor_ids: Sequence[EntityID]now takes a list ofEntityIDs as opposed to indices as was the case foractorsbefore alternatively,actor_types: Sequence[EntityType]allows specifying just the types of entities that can perform the action, without listing individual ids.actors = np.array([1, 2, 3])where 1, 2, 3 happened to be the indices of all entities of some type after joined with all other entity types according to theobs_spaceordering. Now, one could specify this as e.g.actor_ids = [("Robot", 0), ("Robot", 1), ("Robot", 2)]or even justactor_types = ["Robot"]in the case where the action is available to all entities of the "Robot" type.actor_idsandactor_types, in which case all entities can perform the action.SelectEntityActionnow hasactee_idsandactee_typesEnvironment, which implements a maximally convenient interface, andVecEnv, which implements a maximally efficient interface. InEnvironment, actors/actees are referenced via environment-specific ids, andVecEnvreferences everything with indices. The translation happens inListEnv, which converts allEntityIDs into indices when batching observations, keeps track of theEntityIDmapping from the last observation, and converts indices back intoEntityIDs during theactmethod. As a side benefit, this pushes the work of constructing theEntityIDlists into the subprocesses when using parallel envs which allows for additional speedups (I've observed > 22K samples/s now).Smaller changes:
Observation.entitiesrenamed toObservation.featuresObservation.action_masksrenamed toObservation.actionsEnvironment._act|_resetrenamed toEnvironment.act|reset, andEnvironment.act|resetrenamed toEnvironment.act_filter|reset_filter*Batchrenamed toVec*for consistency withVecEnvActiontypes now have separate fields for theactorsandactions/acteesrather than a single list that combines both, and anitems()method that yields an iterator over the(actor, action)pairsEntityType = strandActionType = strto make the intention of the API more obvious.