When training the CAPQL agent, I noticed a drop in the simulation speed when setting checkpoints = True. In the code, if checkpoints = True, the weights of the nets are stored after every step taken by the agent. A parameter such as save_freq could be added to save the net's weights only every save_freq number of global steps.
Another improvement would be to save the weights in different files as its being done in the GPI-PD agent, for example.
When training the CAPQL agent, I noticed a drop in the simulation speed when setting
checkpoints = True
. In the code, ifcheckpoints = True
, the weights of the nets are stored after every step taken by the agent. A parameter such assave_freq
could be added to save the net's weights only everysave_freq
number of global steps.Another improvement would be to save the weights in different files as its being done in the GPI-PD agent, for example.