Potential mismatch between pub_coordination timeframe and `clock.get_step_size()`

[cth127]

I observed that Observation class with an argument timeframe=clock.get_step_size() returns nearly no observation for the pub_coordination environment when deciding a pub. In other words, it cannot consider the conversation part.

As a result, all agents (including built-in baseline agents) return a meaningless Observation and ObservationSummary

I guess there is a mismatch between two arguments, (either clock.get_step_size() is set too short or pub-decision is too late (08:00 AM??)) but I'm not sure.

Is this intended?

[jzleibo]

This is downstream of the general issue currently affecting the together.ai API for the gemma-2-9b-it model. See issue.

This is probably not really an issue with clocks or step sizes. Try it with gemma-2-27b-it.

[cth127]

@jzleibo Well, the Observation part mentioned above does not depend on the LLM call.

[jzleibo]

Does the issue persist when you use gemma-2-27b-it?

[cth127]

Actually, I'm utilizing gemma-2-9b-it locally, not through together api.

[vezhnick]

@cth127, thanks for flagging this.

You can easily adjust the timeframe for the observation and timeframe_delta_from and timeframe_delta_untill for observation summary in the factory class (e.g. rational_agent) or directly in agent_development_tutorial. For the observation it makes sense to capture just the last step. For the observation summary, a longer time frame is sensible and the default we have (4 hours) might be to short indeed. We are going to review the defaults and potentially update the factories.

In the meantime, I recommend experimenting with these parameters yourself. You can, for example, set timeframe for the observation to capture the whole simulation (e.g. 1 year) or even add several summary components covering a multiple scales (hour / day / week).

[cth127]

I see. Thank you for your reply!