anassinator / dqn-obstacle-avoidance

Deep Reinforcement Learning for Fixed-Wing Flight Control with Deep Q-Network
MIT License
74 stars 19 forks source link

Running #1

Open ghost opened 6 years ago

ghost commented 6 years ago

Hi,

I think I have set everything up correctly, where should I run 'director simulator.py' in order to properly produce a result? is their a specific command that should be used? Thanks!

anassinator commented 6 years ago

It should be from the root of this repository, but it is possible this wont run anymore because director and tensorflow APIs were unstable them and have changed a lot since. I'm currently rewriting it to remove the director dependency and use a stable version of tensorflow to help with that.

ghost commented 6 years ago

That might explain... oh great would definitely like to keep updated on that! Will sling you an email @hello@anassinator.com

ghost commented 6 years ago

can i ask when you might be finished with your up to date version? look forward to seeing it

anassinator commented 6 years ago

Sorry I’m on vacation this week. I’m hoping for sometime in the next two weeks.

ghost commented 6 years ago

awesome, enjoy!

ghost commented 6 years ago

Any update on the progress on removing the director dependancy?

anassinator commented 6 years ago

Sorry I've been distracted by some other tasks. So far I have managed to completely replace the director dependency with a simple pygame simulation, but still need to get the rest updated and interfacing with it.

ghost commented 6 years ago

That sounds great, pygame has proved really useful. Any idea when you might complete it? Would love to give it a try.

ghost commented 6 years ago

Managed to get it running by using ubuntu rather than Mac, awesome stuff! Would love to learn more about the theory behind this, have you created any written work I could read on this topic?

anassinator commented 6 years ago

Sorry I took so long to get back to you. I've been busy so I had put this on hold for a while, but I will pick it back up this week. I can't really commit to an ETA to be honest.

I haven't written any paper explicitly on this, but I have outlined this in one section of a larger project. I'm attaching the relevant parts below.

dqn-obstacle-avoidance.pdf

ghost commented 6 years ago

No problem at all. I managed to get the simulator up and running on Ubuntu and I'm amazed by the result, awesome work!

I have 2 quick questions after reading your document. Where might I be able to change the Hyper-parameters such as Learning rate, discount factor within the code if possible? How might I zoom in on the simulator to get a better view?

Thanks!

anassinator commented 6 years ago

When you run director simulator.py --help you will find those options available as arguments to tweak.

As for zooming in, I don't remember and I don't have director set up right now. I believe it was just a mouse wheel or something. Otherwise, you might be ale to find something in director's user guide.

ghost commented 6 years ago

thanks for the help!

ghost commented 6 years ago

One more question sorry!

running 'director simulator.py --help' pulls up the option for arguments for obstacle density, learning rate etc. Apologies if it's obvious but how would I change these values and run?

This work has been very helpful, would love to be able to donate if you have somewhere I can do that!

anassinator commented 6 years ago

No worries! You can add command-line arguments as follows:

director simulator.py --learning-rate 0.1 --exploration 0

The order of the arguments doesn't matter. You would want to set the learning rate and exploration to 0 when done training to get nice results. Ideally, this would simply decrease automatically, but I haven't had time to do that yet.

Thanks for the kind words! I don't have anything set up for personal donations, but this work is being used for a drone we're developing at McGill Robotics (a non-profit student organization) and we will be accepting donations in our upcoming Seeds of Change campaign next week if you're interested. Thanks again!

ghost commented 6 years ago

Great that works well thanks!

I'd be more than happy to make a donation to the campaign, just send me a link! Thanks

ghost commented 6 years ago

Hey! If the opportunity is there I would still like to donate to the campaign as thanks for the help provided!

anassinator commented 6 years ago

Hey @jman123456

Sorry, the campaign was delayed for a bit, but was finally launched here:

https://mcgill.ca/seedsofchange/project/mcgill-robotics

Thanks!