Comments for Self driving toy car using the Asynchronous Advantage Actor-Critic algorithm

[jonjensen]

Comments for https://www.endpointdev.com/blog/2018/08/self-driving-toy-car-using-the-a3c-algorithm/ By Kamil Ciemniewski

To enter a comment:

1. Log in to GitHub
2. Leave a comment on this issue.

[nutte2]

Thank you for nice and interesting code and an excellent description. I tried the code (added some imports and changed the '/' in paths to os.sep) on a Windows 10 machine in Jupyter notebook but got an error.. "BrokenPipeError: [Errno 32] Broken pipe" in ForkingPickler(file, protocol).dump(obj), called via for process in processes: ---> 58 process.start() I guess it might be related to late changes in the multiprocessing library or Windows.. I understand there is no time for finding/searching errors but some hint or maybe info if it can run under Windows would be very useful, thank you. I runned Python 3.7 and PyTorch 1.4 etc.

[kamilc]

Hello @nutte2 - might this be related to this issue?: https://github.com/pytorch/pytorch/issues/4418

We're not using the DataLoader here so there's no place to set the num_worker. We're still using multiprocessing since we need it for the first A in AAA (it's "asynchronous, advantage, actor-critic" after all).

I can't do more digging here - especially that I don't have access to any Windows right now. My next step though would be to look for an alternative threading library that would work fine on Windows.

Hope this helps.

[nutte2]

Thank you @kamilc , yes, it seems related to PyTorch and maybe GPU under Windows..and maybe some recent changes in multiprocessing related to new python-versions and maybe due to Jupyter notebook.. https://medium.com/@grvsinghal/speed-up-your-python-code-using-multiprocessing-on-windows-and-jupyter-or-ipython-2714b49d6fac I can install Linux but don't want to force my students on our Autonomous vehicle university course to do that even if is better :). Or maybe I should :) ? Of course we should support the first A :). Thank you for your quick respone and very impressive code and instructive doc example, will have a deeper look tomorrow (and maybe move some code to a separate file as indicated at medium) !

[kamilc]

Without the first "A" you'd need to use experience replay to de-correlate the updates. You could also "fake" asynchronicity by sampling the trajectories one after another for each runner. This would require a significant rewrite though. The downside would also be a severely increased training time also.

What I'm curious though is: where exactly in PyTorch does it fail? Could you share the full stack trace of the error?

[nutte2]

Thank you, yes, experience replay and double Q-tables were their previous lab, for doing pixel-stuff the first A is very valuable for performance/time and a gift to find your code. I added my tries (.pynb with jupyter notebook and "copy_of" when I tried via Google colab but I guess I failed somewhere with the libraries..) in my https://github.com/nutte2/camil incl the long list of the error, would be very kind if you could have a very quick look at it. (I am from Poland but work in north of Sweden, Umeå, were visiting DL-conf in Warzaw last summer)

[kamilc]

Apologies for stating the obvious about that experience replay. Seems like you've set a really nice curriculum for your students.

As for the errors: I had a quick look and am seeing the following one in the "copy_of" notebook: AttributeError: module 'gym.envs.box2d' has no attribute 'CarRacing'

Could you double-check you have this part of OpenAI Gym installed? (withpip install gym[box2d])

I'm vaguely remembering I needed some external library to make it work. Seems that the following thread in the gym's repo talks specifically about the Windows case: https://github.com/openai/gym/issues/1603

---

Miło widzieć rodaka :) Pozdrawiam

[nutte2]

Thank you (with a little help from google translate ;) ). "Zapomniałem większości polskiego" I now updated some parts/imports in "Copy2.." at https://github.com/nutte2/camil but got stuck on Monitor(). I sounds useful but how to get it? xvbf also complains and a missing stored file..some colab-problem maybe..thank you for your valuable help.

[nutte2]

got the Monitor() to work; a wrapper from gym. now I struggle with colab and xvbf..), found https://colab.research.google.com/drive/18LdlDDT87eb8cCTHZsXyS9ksQPzL3i6H

[nutte2]

Now I got most things to work..but rescale is missing..although I can implement one myself and a read of a non-existent file which I can block. Added Copy3 in my https://github.com/nutte2/camil and also "Run in colab" button so anyone can try to run/change to the better..

[kamilc]

Good to hear! :) That rescale came from scikit-image:

from skimage.transform import rescale