minerllabs / minerl

MineRL Competition for Sample Efficient Reinforcement Learning - Python Package
http://minerl.io/docs/
Other
686 stars 154 forks source link

data_pipelline fails for MineRLNavigateExtreme{,Dense}VectorObf-v0 #364

Open keisuke-nakata opened 4 years ago

keisuke-nakata commented 4 years ago

data_pipelline.py fails for MineRLNavigateExtreme{,Dense}VectorObf-v0.

ERROR    - 2020-07-16 22:41:11,785 - [minerl.data.data_pipeline _load_data_pyfunc 343] Finished

...

  File "/mnt/vol12/nakata/home/minerl2020/minerl2020misc/src/mod/cached_kmeans.py", line 37, in _do_kmeans
    for _, act, _, _, _ in tqdm.tqdm(dat.batch_iter(batch_size=16, seq_len=32, num_epochs=1, preload_buffer_size=32, seed=random_state)):
  File "/usr/local/lib/python3.6/dist-packages/tqdm/std.py", line 1129, in __iter__
    for obj in iterable:
  File "/usr/local/lib/python3.6/dist-packages/minerl/data/data_pipeline.py", line 405, in batch_iter
    for seg_batch in minibatch_gen(traj_iter(), batch_size=batch_size, nsteps=seq_len):
  File "/usr/local/lib/python3.6/dist-packages/minerl/data/util/__init__.py", line 269, in minibatch_gen
    trajs[i] = t = multimap(cat, *[t, next(traj_iter)])
  File "/usr/local/lib/python3.6/dist-packages/minerl/data/data_pipeline.py", line 385, in traj_iter
    s, a, r, sp1, d = trajectory_queue.get()
TypeError: 'NoneType' object is not iterable

I have no idea about the detail, but the error log says [minerl.data.data_pipeline _load_data_pyfunc 343] Finished, which came from https://github.com/minerllabs/minerl/blob/78626361463c42bfc6ba7a69b23ed38990f9811d/minerl/data/data_pipeline.py#L343 It seems that thewhile loop unexpectedly break-ed.

stefanjuang commented 3 years ago

Try dat.batch_iter instead of dat.batch_iter