Learning Agents

[JonasJosef123]

Hello Dear Mesa Community,

i am currently working on a model where agents will be able to learn about other agents' cost functions. Currently each run with my model finished one entire process for all agents after which they get a specific outcome. This outcome will be the input for some form of learning in the next run but now when the batch runner initiates the next run, all my agents' attributes and variables get lost as new agent objects are created. How can i fix this in a smart way? Basically, I need my agent objects to be the same with every batch run.

I am not very experienced with python or mesa, so I am very happy for every help.

Thank you very much!

Best regards,

Jonas

[rht]

If you want determinism, you can specify a random seed to the model (this is not well documented, unfortunately). E.g. https://github.com/projectmesa/mesa/blob/87da503de4a6a77e7f90144b7d43b7fc3d5aeb3e/tests/test_model.py#L29 (but this is not a complete picture either).

What you have to do when initializing the model

class MyModel(Model):
    def __init__(self, var1, var2, seed):
        super().__init__(seed=seed)
        ...

[JonasJosef123]

First let me thank you for your answer @rht. Could you please describe shortly what this seeding process is exactly doing? On google I was only able to learn that a seed somehow helps to run the exact same AB-model with all possible combinations of input variables right? Do you maybe have a good source of information or example where i can better understand how it works?

Again thank you very much!

[JonasJosef123]

I am not sure if i want determinism but rather my agents should be able to save some information (outcome of a whole model run) in their attribute and use that information in another run. But i guess every complete model run will instantiate new Agent objects and therefore delete the information saved in the old agent's attributes. I probably need to completely change my model design so that every model run itself has multiple runs with the same Agents however this will likely raise issues with the datacollector.

Maybe there is another way?

[rht]

Can you share a minimum viable code that reproduces your case? Hard to say anything without concrete code.

[JonasJosef123]

Ok sure:

So I am building a auction model where the Model object acts like an auctioneer and gets a specific amount of objects, the starting price and some AgentParameters from the batch runner:

class AuctionModel(Model):
    """An auction model with some number of agents."""
    def __init__(self, N, objectQuantity, startingPrice, type, AgentParam, seed):
        super().__init__(seed=seed)
        self.num_agents = N
        self.schedule = BaseScheduler(self)
        self.objectQuantity = objectQuantity
        self.currentPrice = startingPrice
        self.type = type
        self.currentBidRound = 1
        self.closed = False
        self.lastBids = []
        self.sortedLastBids = []
        self.running = True
        self.avgPrice = 0

And the Batchrunner from run.py:

fixed_params = {
    "objectQuantity": 1000000,
    "startingPrice" : 15,
    "type": "UVP",
    "AgentParam" : AgentParam(100),
    "seed": 2

}

variable_params = {"N": range(10,40,10)}

batch_run = BatchRunner(
    AuctionModel,
    variable_params,
    fixed_params,
    iterations = 2,
    max_steps = 30,
    agent_reporters = {"Bids": "currentBid","Outcome":"outcome","PricePayed":"payedPrice","VarCost":"variableCost", "FixCost" : "fixCost","History" : "historicBids"},
    model_reporters = {"AvgPrice": AvgPrice}
)

batch_run.run_all()
run_data = batch_run.get_agent_vars_dataframe()

My agents act like bidders. They have a cost function, submit bids and are competing about the objects:

`class AuctionAgent(Agent): """An agent/bidder with fixed utility function."""

def __init__(self, unique_id,AgentParam, model):
    super().__init__(unique_id, model)
    self.unique_id = unique_id
    self.variableCost = AgentParam[unique_id]["variableCost"]
    self.fixCost = AgentParam[unique_id]["fixCost"]
    self.capacity = AgentParam[unique_id]["capacity"]
    self.minimumQuantity = 0
    self.minPrice = 0
    self.historicBids = []
    self.riskBehaviour = ''
    self.experience = 0`

When the model completes one full run, the batchrunner initiates the same model again but with new agent objects. I need the agents to be the same objects as I am saving the auction final outcome in self.historicBids = [] and I want the Agents to use that specific information in this List to change their behavior in the next auction.
I fear there is no other way than calculation multiple auctions within one model instantiation. However this leads to problems with datacollection and the batchrunner as it only safes the last status of the agents attributes in one complete run. Also for me it would be perfect if it works the way i hope because then i could use the batchrunner to run 100 different auctions (for example with different quantities) and easily track the winner of each with the Avgprice etc for example.

I hope this made the problem more clear.

[rht]

BatchRunner is designed for sensitivity analysis, which is different from your use case. That said, you can modify BatchRunner so that you are reusing the same model for new iteration.

Replace https://github.com/projectmesa/mesa/blob/87da503de4a6a77e7f90144b7d43b7fc3d5aeb3e/mesa/batchrunner.py#L165-L166 with

if not hasattr(self, 'model'):
    self.model = self.model_cls(**kwargs)
results = self.run_model(self.model)

The first 2 lines basically initializes self.model once for all.

[JonasJosef123]

First let me thank you for your help again! I have tried your suggestions and they work, but unfortunately the model is not doing what i want it to do. I have the feeling it just copies the results from the first run instead of actually running the model again. I think i will just redesign the whole model and stop using the batchrunner. But thank you very much for your quick answers and help!

[rht]

Oh, maybe you need to reset self.model.running back to be True. I suspect that self.model.running is set to False after first run, and then the model no longer runs.

[rht]

So, before this line

results = self.run_model(self.model)

You do self.model.running = True.

[JonasJosef123]

Hey rht,

I tried your suggestions but my model just stops after one run and i have to use the keyboard interrupt:

Traceback (most recent call last):
  File "run.py", line 44, in <module>
    batch_run.run_all()
  File "C:\Users\Jonas\Anaconda3\lib\site-packages\mesa\batchrunner.py", line 158, in run_all
    self.run_iteration(kwargs, param_values, next(run_count))
  File "C:\Users\Jonas\Anaconda3\lib\site-packages\mesa\batchrunner.py", line 167, in run_iteration
    results = self.run_model(self.model)
  File "C:\Users\Jonas\Anaconda3\lib\site-packages\mesa\batchrunner.py", line 192, in run_model
    model.step()
  File "C:\Users\Jonas\Desktop\Masterarbeit\Model\Auction_Model.py", line 144, in step
    self.datacollector.collect(self)
  File "C:\Users\Jonas\Anaconda3\lib\site-packages\mesa\datacollection.py", line 171, in collect
    self._agent_records[model.schedule.steps] = list(agent_records)
KeyboardInterrupt

[JonasJosef123]

So apparently the process gets stuck in this while loop:

        while model.running and model.schedule.steps < self.max_steps:

            model.step()

Ok i understand know it is moving the model another step. However due to my design my model has closed the auction and stops the process to do anything. Therefore the process stays in infinitely loops in the steps() function. It is not possible that the model refreshes its attributes like object size and starting price and gives another run with dynamically increase new agents while not initializing the old agents (which were initialized in the first run)?

I think i need do rethink my design for my purpose, as you said i use the batchrunner for a different reason than it was designed.

Thanks again for your help!

[rht]

It is not possible that the model refreshes its attributes like object size and starting price and gives another run with dynamically increase new agents while not initializing the old agents (which were initialized in the first run)?

Yes, definitely possible.

I think i need do rethink my design for my purpose, as you said i use the batchrunner for a different reason than it was designed.

You will encounter the same set of problems even if you use something else homegrown that is not batchrunner.