edouardklein / RL-and-IRL

C implementation of RL and IRL algorithms
rdklein.fr/research
19 stars 6 forks source link

Error raised #2

Open Apricock opened 4 years ago

Apricock commented 4 years ago

Hi Mr Klein,

I am a master student in data science from Lancaster University and I am trying to understand the SCIRL and CSI algorithms mentioned in your paper. I tried to run the Exp7.ipynb from in your RL-and-IRL repository, but struggled by some error. I am using python 3.7, and an error occurred in the first part of the isaying that in the 23rd line of stuff.py, arg = arg.reshape((nbinputs,)+input_shape) a float cannot be interpreted as an integer. When I modified the code a little bit into the following, arg = arg.reshape((int(nbinputs),)+input_shape) the first part can run successfully but then in the third part, another error occurred, which is some dimension problem as shown in the following picture,

image

I don’t know whether you have other codes but not yet updated to github or these are problems caused by differences between versions of python? Or did I miss some setting-up?

I am really interested in these algorithms but also much confused. I would be very thankful if you can reply.

edouardklein commented 4 years ago

Hi,

I'm sorry, I haven't touched this code in 7 years, so I'm a bit rusty.

Maybe the commented version can help you out ?

https://github.com/edouardklein/RL-and-IRL/blob/master/tutorials/Exp7.py

I kind of remember that the notebook is basically a dirty version of the tutorial.

I just pushed a fix to stuff.py to correct for your first error message.

When I run tutorials/exp7.py (the python file, not the notebook) I have no errors. Could you try that please ?

In my opinion CSI is superior to SCIRL (but SCIRL got published at NIPS so it is seen as better).

Please do not hesitate if you have any more questions, although I can't promise to answer in a timely manner, I'll do my best :)

I'd be happy to know if you apply any of those algorithm to a real life use case.

Cheers,

Edouard.

Apricock writes:

Hi Mr Klein,

I am a master student in data science from Lancaster University and I am trying to understand the SCIRL and CSI algorithms mentioned in your paper. I tried to run the Exp7.ipynb from in your RL-and-IRL repository, but struggled by some error. I am using python 3.7, and an error occurred in the first part of the isaying that in the 23rd line of stuff.py, arg = arg.reshape((nbinputs,)+input_shape) a float cannot be interpreted as an integer. When I modified the code a little bit into the following, arg = arg.reshape((int(nbinputs),)+input_shape) the first part can run successfully but then in the third part, another error occurred, which is some dimension problem as shown in the following picture,

image

I don’t know whether you have other codes but not yet updated to github or these are problems caused by differences between versions of python? Or did I miss some setting-up?

I am really interested in these algorithms but also much confused. I would be very thankful if you can reply.

edouardklein commented 4 years ago

Also, I would advise you to use the problems at https://gym.openai.com/ for testing. It did not exist back when I wrote this code, but it is the standard way of testing RL and IRL algorithms on toy problem now.

You'll find the mountain car exemple there https://gym.openai.com/envs/MountainCar-v0/

Please tell me if the proposed solution works for you so that I can close the issue.