Caching from agent object

[Chan-Dong-Jun]

What's the problem this feature will solve? Previously, objects were cached from grid objects, and the visualization module was changed to read parquet files. rht advised that it is not modular. The new method caches the agent object and recreates the model object so that the visualization module does not have to be altered to take in parquet file.

Describe the solution you'd like

1. Cache the positions of the agents using agent objects.
2. Recreate the model objects with the grid positions so that they can be passed into the visualization module.

[Chan-Dong-Jun]

Main edit

Caching methods

        def get_data(self, parameters):
        agent_dict = list(self.model.agents_.values())
        complete_row_data_deserialized = []
        for key, val in agent_dict[0].items():
            complete_row_data_deserialized.append(key.__dict__)
        boids_data = []
        for row_record in complete_row_data_deserialized:
            clean_row_data = {}
            for param in parameters:
                clean_row_data[param] = row_record[param]
            boids_data.append(clean_row_data)
        boids_table = pa.Table.from_pylist(boids_data)
        padding = len(str(self._total_steps)) - 1

        filename = f"{self.cache_file_path}/grid_data_{self.model._steps:0{padding}}.parquet"

        pq.write_table(boids_table, filename)
        return boids_data

This caches the agent object into parquet files.

[Chan-Dong-Jun]

Main edit

Recreating model object

reconstructed_model = mesa.Model()
column_list = df.columns
agent_list = []
for idx, row in df.iterrows():
    agent = Boid(None, reconstructed_model, None, None, None, None)
    for column in column_list:
        setattr(agent, column, row[column])
    agent_list.append(agent)

This method creates the object and uses setattr to set the attributes of the modle objects from cached data.

[Chan-Dong-Jun]

The above is tested with boid flockers and wealth model

[rht]

        agent_dict = list(self.model.agents_.values())

This seems outdated. The latest main has 3 objects to store agents. I think the agents_by_type one should be used instead, so that we only survey the attributes for each types once for all, instead of every single agent.

        for key, val in agent_dict[0].items():
            complete_row_data_deserialized.append(key.__dict__)

This works only if there is 1 type of agent. For the wolf-sheep-grass example (3 types of agents), the grass has a different set of attributes from the wolf and the sheep.

Would be informative to have a benchmark of how long it takes to create the cache, restore the cache, and step. So that we can track progress in the performance optimizations.

[tpike3]

@Chan-Dong-Jun What version are you anchoring to?

To @rht's point you could also use agent_types property to ensure you get all agents in agents