This was intended to create a parallel training environment. It has gotten pretty stale now, and I am not sure it is entirely feasible. The ray training seems to be the simplest way to implement it, however I havent had much success.
The value of this training would still be quite high, since our current training is rather slow. Since starting my last run as of 8/12, started on 8/10, we have trained ep 44, and ep 10 (which is about the time, if not a good while after, that we start replay training, which takes up the bulk of the processing time) was 48 hours ago. Meaning we train 0.7 episodes/hour. So trying to train 10,000 episodes would take 587 days, so like a year and a half. Might have to consider buying some server time afterall, but id rather try to get something parallel working, because that would make the best value for the time as we can get.
Might revisit this later. But it may be more value to start a new PR from scratch due to the many changes made since this one, or at the very least try to implement some other time saving measures that are less intense.
This was intended to create a parallel training environment. It has gotten pretty stale now, and I am not sure it is entirely feasible. The ray training seems to be the simplest way to implement it, however I havent had much success.
The value of this training would still be quite high, since our current training is rather slow. Since starting my last run as of 8/12, started on 8/10, we have trained ep 44, and ep 10 (which is about the time, if not a good while after, that we start replay training, which takes up the bulk of the processing time) was 48 hours ago. Meaning we train 0.7 episodes/hour. So trying to train 10,000 episodes would take 587 days, so like a year and a half. Might have to consider buying some server time afterall, but id rather try to get something parallel working, because that would make the best value for the time as we can get.
Might revisit this later. But it may be more value to start a new PR from scratch due to the many changes made since this one, or at the very least try to implement some other time saving measures that are less intense.