search

glistering96 / AlphaRouter

2 stars 0 forks source link

Numpy vector env for training phase #6

Closed glistering96 closed 1 year ago

glistering96 commented 1 year ago

To boost up the training, numpy vector based environment seems to be better.

I have implemented it on the branch np_vec_env with a test bed.

Will merge into the master if it is validated to work ok.

glistering96 commented 1 year ago

I've found a severe bug when using Gymnasium's SyncVecEnv, in which the episode is not terminated. #3 is wrong and slow compared to the numpy vec environment. So I just remove the gymnasium Sync Vec Env.

Note that numpy TSP env runs around x2 faster than the SyncVecEnv