search

cogment / cogment-verse

Research platform for Human-in-the-loop learning (HILL) & Multi-Agent Reinforcement Learning (MARL)
https://cogment.ai/cogment_verse
Apache License 2.0
80 stars 15 forks source link

Fix DQN model update frequency #127

Closed wduguay-air closed 1 year ago

wduguay-air commented 1 year ago

Context

The actor_session.get_tick_id() resets back to 0 after every trial. For environments where the number of steps is lower than the config.model_update_frequency, the model update condition will be skipped.

Solution

Add a sample counter for the Actor class.

Other changes

Improve readability of dqn configs.

cloderic commented 1 year ago

@wduguay-air can you remove the counter ? as discussed it will cause issues.

wduguay-air commented 1 year ago

Removed here: https://github.com/cogment/cogment-verse/pull/128

wduguay-air commented 1 year ago

I will try to find time this week to implement the complete solution with the model registry v2.