search

TimZaman / dotaclient

distributed RL spaghetti al arabiata
26 stars 7 forks source link

Merge DotaWorld #17

Closed Nostrademous closed 5 years ago

Nostrademous commented 5 years ago

Issue #9

This also adds initial ability_leveling policy updates.

I worry we are diverging too much.

Nostrademous commented 5 years ago

Just added tracking of tower units for enemies and allies.

Also, tested ability leveling (by forcing it to always level the first option). Not sure how to implement it in torch sensibly. I create a int32 Tensor for it but not sure how to determine selection in a ML way.

Nostrademous commented 5 years ago

@TimZaman what do you want to do with this PR? I was going to start implementing ability affines but also think we need to greatly improve the basic_unit info provided.

I really would like to move over to the dotaworld use since it creates the necessary classes to enable tracking the data we need.

Thoughts?

Nostrademous commented 5 years ago

rebased and force-pushed this PR after your latest commits

Nostrademous commented 5 years ago

Okay, you made a lot of changes and you don't seem to want this monolithic PR. Let me remove it and make new PRs for various pieces with your latest code in ideally individual branches so as not to mix various idaes.

TimZaman commented 5 years ago

Yeah plz make small ones. I do need to stay agile! What i have now is quite nice though.

On Mon, Jan 21, 2019, 09:51 Nostrademous <notifications@github.com wrote:

Okay, you made a lot of changes and you don't seem to want this monolithic PR. Let me remove it and make new PRs for various pieces with your latest code in ideally individual branches so as not to mix various idaes.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/TimZaman/dotaclient/pull/17#issuecomment-456154559, or mute the thread https://github.com/notifications/unsubscribe-auth/AHXSRKVwimhF_o156JdiF5cGrYfpBVRnks5vFf4igaJpZM4aFvSq .

Nostrademous commented 5 years ago

I agree, but I think we are shooting ourselves in the foot a bit by not including certain things to help the agent learn.

For example - attack_target_handle & ability_target_handle for basic_unit affine. Without those it is hard for the agent to learn why it is taking damage... sure, based on HP it can figure out that it is taking damage, but not why.

TimZaman commented 5 years ago

Correct, it doesnt yet see incoming damage or projectiles.

On Mon, Jan 21, 2019, 10:18 Nostrademous <notifications@github.com wrote:

I agree, but I think we are shooting ourselves in the foot a bit by not including certain things to help the agent learn.

For example - attack_target_handle & ability_target_handle for basic_unit affine. Without those it is hard for the agent to learn why it is taking damage... sure, based on HP it can figure out that it is taking damage, but not why.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/TimZaman/dotaclient/pull/17#issuecomment-456161192, or mute the thread https://github.com/notifications/unsubscribe-auth/AHXSRAwVQDhtN9tbEIy_aYm0o-Oclq_7ks5vFgSNgaJpZM4aFvSq .