search

liquidcarrot / carrot

🥕 Evolutionary Neural Networks in JavaScript
https://liquidcarrot.io/carrot/
MIT License
295 stars 34 forks source link

Fixing the DDPG agent #206

Closed raimannma closed 4 years ago

raimannma commented 4 years ago

Here comes a fix for the DDPG agent

christianechevarria commented 4 years ago

Hey @raimannma thanks for updating the branch! Not sure if you got a chance to look at the TravisCI build but it seems like there's some long-lived process in the unit tests that is logging and the build was unable to finish. Would you be able to look into why that's happening?

raimannma commented 4 years ago

Oh, this seems to be just a timeout problem.

My PC is so much faster than the travis-ci server. I will do some tests and change the timeout.

raimannma commented 4 years ago

if I test it locally, everything works just fine.

Can you retry the travis build?

christianechevarria commented 4 years ago

Hey @raimannma, just tried to re-run the Travis build but wasn't able to, I think Travis has a time limit to re-run builds. Any chance you could make a tiny chance and push to the branch? If possible removing the console logging could make the test run much faster

raimannma commented 4 years ago

Hey, I'm sorry, I think there is still a issue. 9 of 10 tries it works perfect. But sometimes the agent makes the same actions, although he knows that they are bad, from previous experience.

I am not sure where the bug is, probably it's the noise function.

christianechevarria commented 4 years ago

I am not sure where the bug is, probably it's the noise function.

Yeah, I've found that debugging ML code is often really tough. Even with unit tests isolating where things happen is usually a tall order. That's part of the reason I've been trying to work on a way to log events for NNs in a Redux-style so you can see whenever changes happen to the network structure / configuration and step forwards / backwards in time.

For now what about trying to just log everything and follow the execution of the code? Maybe that could help to find the bug

raimannma commented 4 years ago

There is an issue with the target networks. I will do this whoule reinforcement things in the new typescript version. So, for now closing this pull request.