Env reset issue - Githubissues

Titouaaaan commented 2 weeks ago

Hello,

Ive been experimenting with this amazing library but i am encountering a bug which stops me from training agents. After succesfully making the env, the env seems to get stuck at the reset. The game window is properly recognized, the player gets teleported to the arena but iudex and the player just stare at each other. The only way to trigger the encounter is by manually pressing q (lockon) on the keyboard, and after that the random agent works perfectly fine. Ive tried all the other keys but it is the only one that works.

To resolve this, i tried using virtual key presses as well as using the GameInput class to add actions to the queue (i even did multi threading to spam q until the env manages to reset but that didn't work). I also run the code in administrator mode.

I am using Python 3.11.2 and Soulsgym 1.1.0. I'm also running DSIII on steam on version 1.15.2.

Thanks in advance for any help :)

amacati commented 2 weeks ago

Hi, I am not sure what is causing this bug as I can't reproduce it on my side. Have you installed the library via pip or directly from the repository? In the latter case, you could try running the dev branch and see if that fixes any issues. And just to make sure, the rest of your key setup is exactly as mentioned in the docs?

Titouaaaan commented 2 weeks ago

Hello, i installed via pip but i cant try installing directly from the repo and see if it changes anything on the dev branch. Concerning my setup, i binded all the keys correctly as well as the graphics settings (and im in offline mode) so im not sure what i could be missing... Ill test the dev branch and let you know if it changes anything! Thanks a lot for your help :)

Titouaaaan commented 1 week ago

Hello again,

So i double checked my setup and all the versions are correct as well as the key bindings and the graphics. I ran the test.py of the dev branch, but i get the same problem, the fight doesn't start (i.e no reset) until i press q. I will try uninstalling everything and installing again but I really don't see where the problem could be coming from. If you have any ideas of what i could have missed in the setup I would be really grateful. Also if you need any information of my setup I can provide that.

Thanks in advance :)

amacati commented 1 week ago

This is puzzling. Can you maybe let a keylogger run in the background and check if 'q' is being registered as a press in general? Also, could you check what happens after you press 'q' twice? I.e. is autolocking working after you have pressed 'q' once?

Titouaaaan commented 6 days ago

Ok i tested a few things, The encounter starts only when 'q' is pressed manually and if i manually lock off after the encounter starts (and no inputs from me or agent after) the env wont reset (normally it should twice in the test.py file). Howver, if i stay locked on during the fight and die, since i am still locked on the env manages to reset (like it should). When i do a random agent, the agent does manage to lock on and off when it presses 'q' (so autolocking works), but i can only test that after presing 'q' manually.

Could you maybe explain how you reset the env? Because i tried looking into it but if you explain i might be able to understand what the issue is (also i'll run a keylogger in the afternoon and keep you posted). Thanks a lot for the help

amacati commented 5 days ago

It's implemented here: https://github.com/amacati/SoulsGym/blob/455118374e3322f2cc748a1b54894f103a4b797d/soulsgym/envs/darksouls3/iudex.py#L192-L224 reset():

Resets your keyboard buttons to make sure nothing is being pressed atm
Focuses the game window
Reloads the game if flags are false and periodically to prevent weird behavior after long training times
Teleports to the arena and enters the fog wall
If in phase 2, sets Iudex HP to < 50% and waits for his phase transition
Resets the player and boss entities to full HP/SP/MP and teleports them to their start positions
Resets the camera to lock on to Iudex
Pauses the game and sets the appropriate game flags

So the problem is very likely with IudexEnv._camera_reset(), which is implemented in https://github.com/amacati/SoulsGym/blob/455118374e3322f2cc748a1b54894f103a4b797d/soulsgym/envs/darksouls3/iudex.py#L326-L335 You could check if the first lock on fails and 'q' remains pressed. In theory, the reset in line 333 should prevent that from happening, but that's the most likely failure mode in my opinion

Titouaaaan commented 4 days ago

Ok i finally found what the problem was: the agent was not able to lock on because for some reason, the input sent to the game was using a different keyboard layout than the one I set. I am using a qwerty keyboard and my language setting is english. But the secondary language is french on my pc and it seems like the env was virtually pressing on a azerty keyboard and not qwerty. I realized that because after a bunch of testing, when i forced the agent to press only 'q', it would walk to the left (and q is where a -the left input- is on a azerty keyboard). The keylogger helped for debugging that thanks a lot. I still don't know why it behaved that way since all my settings are in english and use english keyboard. I reinstalled the library on the main branch and it works as intended, now that i removed the french language setting. Thanks a lot for your time, your library works perfectly :)

(PS: i don't know if this has happenned before but maybe a little disclaimer in the documentation to indicate that having multiple language settings could interfere with the game input might help in the future)

amacati commented 1 day ago

Thanks for looking into this! I have updated the docs accordingly.

amacati / SoulsGym

Env reset issue #49