Closed Nostrademous closed 5 years ago
Yeah i should add the valid-action-mask itself to the forward function, because i should run the softmax again over the invalid actions.
I did this now. thanks
I feel like the bot is learning much faster thanks to this. This is based on purely watching the bot and its decision making.
What do you think?
Yep very true
On Fri, Feb 1, 2019, 06:16 Nostrademous <notifications@github.com wrote:
I feel like the bot is learning much faster thanks to this. This is based on purely watching the bot and its decision making.
What do you think?
— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/TimZaman/dotaclient/issues/14#issuecomment-459736235, or mute the thread https://github.com/notifications/unsubscribe-auth/AHXSRD1ZI2sGJidkKtFNLNMiNf-_7ZyKks5vJEw2gaJpZM4aBi98 .
I thought you were going to increase the reward values for Win/Loss? I still see kills/deaths as more impactful in current version.
Doesnt matter at the moment, it doesnt explore or train well. It gets to be a last hit master but there is no smart play going on.
On Fri, Feb 1, 2019, 09:38 Nostrademous <notifications@github.com wrote:
I thought you were going to increase the reward values for Win/Loss? I still see kills/deaths as more impactful in current version.
— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/TimZaman/dotaclient/issues/14#issuecomment-459803900, or mute the thread https://github.com/notifications/unsubscribe-auth/AHXSRIzljQ8NGxXswIYhQ5N-tkFu48Nkks5vJHt6gaJpZM4aBi98 .
So I think I'm going to work on fixing that by doing several things:
1) Teach it about lane fronts and reward it by being near it (rather than at location (0,0,0) <--- center of map 2) Change the Tower HP reward to be a delta between the Enemy Tower HP and Friendly Tower HP (as in "it's okay if our tower is taking damage providing we are doing more damage to enemy tower")
It should help.
Separate from that, I'm working on formalizing a new ML approach to how to teach agents to do strategic planning versus tactical actions. I hopefully can share with folks outside my company.
Actually the distance reward can be removed. I just sent you an email, ill be at OpenAI tomorrow. Any questions we have for them?
On Fri, Feb 1, 2019, 10:19 Nostrademous <notifications@github.com wrote:
So I think I'm going to work on fixing that by doing several things:
- Teach it about lane fronts and reward it by being near it (rather than at location (0,0,0) <--- center of map
- Change the Tower HP reward to be a delta between the Enemy Tower HP and Friendly Tower HP (as in "it's okay if our tower is taking damage providing we are doing more damage to enemy tower")
It should help.
Separate from that, I'm working on formalizing a new ML approach to how to teach agents to do strategic planning versus tactical actions. I hopefully can share with folks outside my company.
— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/TimZaman/dotaclient/issues/14#issuecomment-459817907, or mute the thread https://github.com/notifications/unsubscribe-auth/AHXSRJ_-4cUG2y_Osbxiisgphhs6Z-l-ks5vJIVHgaJpZM4aBi98 .
Might be easier to chat briefly. Jump into a Google Hangouts if you can.
I'll be sitting there for the next 20-30min.
LINK DOWN
It was sad and lonely.... replied to you email.
So looking at code - you don't disable all "invalid" actions.
The code only prevents you attacking yourself, or issuing an attack action all together if there are no valid unit_handles, it does not prevent attacking your own units if they are at full health (which has no effect, although technically valid).
I'm not saying this is a problem that needs fixing, just pointing it out. Attacking your own units at full health is a way to drop tower aggro for example.
Sure, i knew that. In principle its just good to prevent. I do need to check what is going on when the hero is dead.
On Mon, Feb 4, 2019, 07:47 Nostrademous <notifications@github.com wrote:
So looking at code - you don't disable all "invalid" actions.
The code only prevents you attacking yourself, or issuing an attack action all together if there are no valid unit_handles, it does not prevent attacking your own units if they are at full health (which has no effect, although technically valid).
I'm not saying this is a problem that needs fixing, just pointing it out. Attacking your own units at full health is a way to drop tower aggro for example.
— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/TimZaman/dotaclient/issues/14#issuecomment-460297947, or mute the thread https://github.com/notifications/unsubscribe-auth/AHXSRKsUNp1ddnkpnQ2JlM2NWj-Vgv83ks5vKFYWgaJpZM4aBi98 .
One suggestions that seems reasonable to me is: https://ai.stackexchange.com/a/2994
This is regarding: https://github.com/TimZaman/dotaclient/blob/master/policy.py#L127-L133