ocram444 / EldenRL

Reinforcement Learning for Elden Ring on Windows11
GNU General Public License v3.0
23 stars 10 forks source link

Question with PvP training #1

Open xliu21-5 opened 11 months ago

xliu21-5 commented 11 months ago

I find this project very interesting and I have a few questions and concerns about the pvp training part.

  1. The training requires a lot of dueling samples for training, and arena is quite slow. It's possible to get more duels by using the seamless coop mod and auto revive script with CE. This would require changing the duel start condition and win conditions. I wonder if these changes could be made possible. The start condition can perhaps be of lockable target is detected, as long as we set the character at a fixed location at a fixed angle at the start.

  2. Is it possible to add certain action sequence as preconditions? Say using wonderous physik and setting character to set position as I wondered in point 1

  3. Given how many pvp builds and setups are available, I wonder if we can set the model to randomly choose from different setups and builds before the duel

  4. I have thoughts on what other reward/penalty conditions can be added, but I think other members in the pvp community will have different opinions as well. Would you mind if I introduce this project into the elden ring pvp community and tweak the reward/penalty with crowd wisdom?

ocram444 commented 11 months ago

Hey, Thanks for being interested in the project :)

Why don't we continue this discussion on the EldenBot discord server ( https://discord.com/channels/984553640071155752/984553641815982140)? That's where most of the conversation about the project happens.

But let me see if I can address your thoughts now.

1.1 I'm not familiar with the seamless coop for dueling but from what I've been able to look at quickly it should be possible to use it. If you can repeat duels against the same enemy endlessly (maybe even on one computer, or ai against ai) that would drastically increase the amounts of duels we could run as it would cut out matchmaking entirely.

1.2 Not quite sure on how to define the start (or end) condition though. Currently it works by detecting a win or a loss either by the duel winning message on screen or a player death, then after matchmaking and the loading screen we can lock on the enemy and let the ai take over. Could be difficult without a message letting the system know it just killed an enemy. But certainly not impossible.

2.1 Action sequences are possible for sure. The ai can take actions like running-attack or jumping-attack which combine a couple of key presses.

2.2 Walking to a specific position could be difficult again. Teleporting to a bonfire allows you to get a consistent starting position but defining a position to walk to based on coordinates or something isn't implemented in the project as it stands.

  1. Random builds are a cool idea I've looked into before. I thought about letting twitch chat equip items to the character with channel point rewards. But you need to have a big enough channel to be allowed to use channel point rewards... Cool idea though and definitely possible.

  2. Introducing the project to more people sounds great! Setting rewards and reward amounts is an important part of RL systems and could help the agent learn specific things we want to reward/punish. Always great to have more people thinking about it.

Hope that helps, let me know if you need any more input and feel free to share any progress on the discord server.

xliu21-5 commented 11 months ago

Glad you responded so quickly. The discord link you shared doesn’t seem to work, could you double check?

ocram444 commented 11 months ago

Ah sorry, hers the link https://discord.gg/nThwu88Q

On Mon, Nov 6, 2023 at 5:21 PM x21L5 @.***> wrote:

Glad you responded so quickly. The discord link you shared doesn’t seem to work, could you double check?

— Reply to this email directly, view it on GitHub https://github.com/ocram444/EldenRL/issues/1#issuecomment-1795339181, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3C7Z7IFXIWYY2ONAJWHLIDYDEFCJAVCNFSM6AAAAAA67BUFWGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJVGMZTSMJYGE . You are receiving this because you commented.Message ID: @.***>