search

toshikwa / fqf-iqn-qrdqn.pytorch

PyTorch implementation of FQF, IQN and QR-DQN.
MIT License
161 stars 24 forks source link

A function about continuing training #13

Open Hugh-Cai opened 3 years ago

Hugh-Cai commented 3 years ago

Hi, guys! I often have a problem that when I train the agent in the computer, the code is terminated. So, I think if it can be realized that the agent terminated unexpectedly can continue to complete the training steps.

And another question, how long does it take to train for 200M frames per game?

thanks

toshikwa commented 3 years ago

@Hugh-Cai

Thank you for the suggestion, that would be great ;) I'd love to implement that function, but I have no time for that now... I will implement it in two weeks. Or do you want to contribute?

And another question, how long does it take to train for 200M frames per game?

Around between 6 days and 8 days, depending on the env, algorithm, and computational resources.

Thanks.

Hugh-Cai commented 3 years ago

@ku2482

I’m glad you can take my proposal,

Or do you want to contribute?

I once wanted to implement this function when I reproduce the Rainbow, but failed due to my weak programming ability (drying my tears...). My simple thought is to record the training step and other parameters in .csv, and then read them when loading the re-trained model. However, it doesn't work because of the lack of environment interaction. I don't know how to keep information about the environment like ALE or its extension gym and to reuse them.

Sorry.