search

germain-hug / Deep-RL-Keras

Keras Implementation of popular Deep RL Algorithms (A3C, DDQN, DDPG, Dueling DDQN)
528 stars 149 forks source link

Can not run DDQN with PER #21

Open Xiaoyu006 opened 4 years ago

Xiaoyu006 commented 4 years ago

Hi,

Thanks for this great implementation. I encountered an error when tried to run "python3 main.py --type DDQN --env CartPole-v1 --batch_size 64 --with_PER"

The error is: /utils/memory_buffer.py", line 69, in sample_batch batch.append((*data, idx)) TypeError: 'int' object is not iterable

Have anyone fixed this issue?

thinhlx1993 commented 4 years ago

Hi,

Thanks for this great implementation. I encountered an error when tried to run "python3 main.py --type DDQN --env CartPole-v1 --batch_size 64 --with_PER"

The error is: /utils/memory_buffer.py", line 69, in sample_batch batch.append((*data, idx)) TypeError: 'int' object is not iterable

Have anyone fixed this issue?

I got the same issues. Did anyone fixed it?

surfingkaka commented 4 years ago

trying passing in the data directly instead of the *data I think the author were attempting to make it future proof.

Jeosas commented 4 years ago

Hi, I'm having the same issue.

Changing *data to data bring an error on the next line idx = np.array([i[5] for i in batch]) : tuple index out of range

I've done a little debugging by printing out the the (idx, data) values and something stood out: " 22014 (array([-0.08095277, 0.61171632, 0.0524595 , -0.81013508]), 1, 1.0, False, array([-0.06871845, 0.80608178, 0.0362568 , -1.08586616])) 22015 (array([-0.06871845, 0.80608178, 0.0362568 , -1.08586616]), 0, 1.0, False, array([-0.05259681, 0.61050081, 0.01453948, -0.78203035])) 22017 (array([-0.04038679, 0.80541993, -0.00110113, -1.07010368]), 1, 1.0, False, array([-0.0242784 , 1.00055643, -0.0225032 , -1.36313198])) 22017 (array([-0.04038679, 0.80541993, -0.00110113, -1.07010368]), 1, 1.0, False, array([-0.0242784 , 1.00055643, -0.0225032 , -1.36313198])) 22018 (array([-0.0242784 , 1.00055643, -0.0225032 , -1.36313198]), 1, 1.0, False, array([-0.00426727, 1.19595297, -0.04976584, -1.66276796])) 22019 (array([-0.00426727, 1.19595297, -0.04976584, -1.66276796]), 1, 1.0, False, array([ 0.01965179, 1.39161785, -0.0830212 , -1.97052703])) 32767 0 "

For some reason, the get function of the MemoryBuffer return "int(0)" as data, that's what's causing the "tuple index out of range" error. We can also see the faulty idx which jumps to a crazy high value for some reason, and thus out of range of the memory i think (ending up in the data being a 0 ?) EDIT: idx ends up pointing to a memory slot that has not yet been written (thus returning 0)

I'm still investigating but any help would be appreciated :)

winston-p commented 3 years ago

Hi, has anyone managed to fix this bug? I am new to DDQN and hit into this issue. Hope to get some help. Thanks in advance!