Naylepsh / guild-wars-2-auto-daily-rewards

Automate daily login rewards acquisition for multiple accounts
3 stars 1 forks source link

Problems & Addition for Readme "How to install/use" #5

Open lukycody opened 1 year ago

lukycody commented 1 year ago

Hey there. So I tested quite some stuff and want to give you feedback and problems I encoutnered.

First off I tried to run this and it turned out for me, that I first needed to install some packages in my vscode: py -m pip install pyautogui py -m pip install pyyaml

May be worth it to point out in the readme description. Bonus questions just out of curiosity to better understand code: Why do you need this poetry thing in the first place (packet manager? what does it offer here?) and why do you need to start the main.py script via poetry aswell? (poetry run python main.py)

Problem with the script: Besides that the script is not working for me as it should. The problem is that first the login credentials don't get entered correctly, because the focus/position of the pyautogui is not right. Then the focus swaps to another window (which e.g. is the vscode window) and tries to enter the credentials there, which messes up the whole thing being not in sync.

  1. When the launcher starts with previously ticked boxes to save credentials, the focus is on the password field so it seems to mess up the login() function and its credential input. --> fixed that by deleting previous credentials and unticking the boxes

  2. However, the script also tabs out to another window and tries to insert credentials there, not in the launcher fields. Not sure why its tabbing out at all though, as that behaviour doesnt seem to be in the login() function. Also I'm not sure, why you use double_click() before clearing the input. Maybe this klicking at current position creates the tabbing behaviour (or are you supposed to not even touch the mouse after running the script?) But I also think that you have to select the correct fields for email and password first (pyautogui.moveTo(x,y)) before trying to clear_input().

Edit: it clearly moves the mouse to a spot where theres no launcher/field to click. My launcher always starts right in the middle of the screen, but the cursor moves to the upper left direction and clicks there. So theres something to fix about the positioning of either the cursor coordinates or the launcher (probably should manually set both positions in the code to sync the actions).

Edit2: it just came to my mind: are the coordinates maybe set for FHD 1920x1080 resolution? I'm using 2560x1440 atm, maybe the reason. If thats the reason you could maybe add a request for a user input of the used resolution at programm start and set the coordinates accordingly to that. --> that seems to "kinda work" with FHD. Sometimes the mouse position gets on the input field, sometimes it still randomly "tabs out" somewhere else.

Edit3: Another bug in the code; This doesn't work with mails that have a "+" in them. It gets replaced by "*" which ofc then causes the login to fail. Idk why it does so, as it should take the original text from the .yaml and just pass it 1:1 as a string not replacing anything. That actually breaks the whole process if youre using such an email address.

Naylepsh commented 1 year ago

About the installation -- there's already a section "How to run" in the README which mentions poetry install, which, well, installs the required packages.

About the "Why do you need this poetry thing" -- it's a package manager. IMO it's a better way of handling dependencies than the questionable standard of venv / pyenv, pip & requirements.txt trio.

About the "why do you need to start the main.py script via poetry" -- so that the script has access to the dependencies. Similarly, if you were not to use poetry, but the usual pip and requirements.txt, before running the script you'd have execute source ./venv/bin/activate, so that the script can shift to the environment where these dependencies are present. Or you could skip this step and install packages globally instead, but that's... not exactly maintainable.

About the "are the coordinates maybe set for FHD 1920x1080 resolution?" -- yes. The README mentions that currently only FHD resolution in supported. Because my screen doesn't support higher resolutions, I'm not able to test other coordinates. However, if you were to input your own and later send a MR request (or at least send me a list of coordinates) -- that would be appreciated. And to the related "you could maybe add a request for a user input of the used resolution at programm start and set the coordinates accordingly to that" -- that's a good suggestion, but I need the coordinates for other resolutions first lol.

I'll address the other questions / suggestions / issues later.

lukycody commented 1 year ago

Thanks for the quick reply and explanation on the poetry usage. Sorry, totally missed that assumption of the resolution, duh! If I find a way to figure out the coordinates of the fields/buttons on screen I‘ll test it on my resolutions up to WQHD! (Can’t test 4K though) and let you know! :) (never did that, how did you grab them manually?) To add onto the idea of changing coordinates accordingly to the users resolution: you also could read the windows position at start and address the fields/buttons relative from the windows boundaries, instead of hardcoding the coordinates for different resolutions (not sure if the window scales too though).

About the Readme part: I saw that and followed the advice. ‘poetry install‘ didn‘t make it for me though. Executing the script gave the error of missing packages. Somehow, I had to manually install the pyautogui and pyyaml dependencies via pip, after that it worked.

Only barrier for further testing now is the replacement of + vs * in the credential text. As a workaround it should be possible to build in a custom string slice and replacement step before actually pasting it into the field, but that part with credential reading from the yaml file and storing in that dataclass/struct already was a bit mystical to me - and I suppose for you too („#noqa“). However, I feel like that shouldn’t be necessary, just gotta understand why it gets replaced. Probably the encoding?

Naylepsh commented 1 year ago

"(never did that, how did you grab them manually?" -- I just make a screenshot, then open it in paint. Then just hover a cursor over the parts I'm interested in. It displays the coordinates in the bottom left corner.

"Executing the script gave the error of missing packages" -- How have you installed poetry? Through pip or through official installer? The pip way is known to cause some issues sometimes.

"Only barrier for further testing now is the replacement of + vs in the credential text." -- well, this is weird. I just tried on my end and using "+" in the yaml's email field worked fine lol. Both the extraction from file and passing it to input passed without any issues. Is this happening with any other characters on your end? Or just the "+" -> ""? On one hand it feels like some encoding issue, but on the other, it's explicitly set to "utf-8" -- it should be treated properly. This sure is scuffed.

And about the "#noqa" -- that doesn't mean I haven't done QA on my end, it's just a tag for the linting / formatting tools to ignore this line. Otherwise they're gonna complain that the line is too long. But I cannot make it shorter, because lambdas cannot be longer than 1 line.

Naylepsh commented 1 year ago

A thought about this "+" -> "*" issue -- can you try to print out your credential's email field in the login function and check whether that matches what you have inputted? I wonder whether the extracting data from YAML works fine, but it's pyautogui.write that causes the issue.

lukycody commented 1 year ago

Ah thats a neat little trick with paint! Here are the positions/coodinates of the launcher fields (x,y) for WQHD resolution (2560 x 1440):

Yep, I did the official install method for windows from their website, just as you stated in the readme.

About the "+" -> "*" issue: It prints the correct data inside login(). So there seems to be something wrong with pyautogui indeed. Any idea how to fix this? And yes, it's only this one character that gets replaced. Only .+@ used as "special characters".

noqa ... I just googled it... quality assurance.

Oh wow, I thought #noqa would mean "no questions asked" because you also used it first with the following comment in the code:

"# For whatever reason double clicking through pyautogui is not enough, but more clicks somehow works # noqa"

It's like "huh whatever, if it, works it works. no questions asked"😂

Naylepsh commented 1 year ago

"#noqa" -- Ha, I was looking on a different line when I was thinking about it (get_credentials and not the enforce_click). But the idea's the same -- line is too long.

About the coordinates -- nice, thank you. I'll look into adding those.

And finally about "+" -> "*" -- It might be that you're using some non standard keyboard layout. For example, when pressing alt+l normally nothing happens, but when I change the layout by pressing ctrl+shift, then the same alt+l operation now yields ł character. Similar thing you might had seen in laptops is than after pressing some functional key, pressing some arbitrary element on keyboard decreases / increases volume or brightness, etc. From stack overflow https://stackoverflow.com/questions/73193008/why-is-pyautogui-replacing-with : "This usually happens due to the use of non-qwerty keyboards. When you use write (or press), pyautogui doesn't actually write that character, instead it looks for the associated key on your keyboard and presses it"

lukycody commented 1 year ago

Well I‘m using an ISO DE standard keyboard. Hm, would it suffice to force swap to the EN layout in windows before running the script (or even doing that programmatically) to solve this issue. Or do you actually need the physical periphery for that? That’s weird honestly. It‘d mean, that people with another keyboard other than ansi EN layout can’t reliably use pyautogui 😂

Naylepsh commented 1 year ago

Theoretically switching to EN layout should be enough (well, I hope at least lol). In that stack's post it was also mentioned that there are "hacks" to make it work -- like writing only standard chars, but whenever seeing those special symbols then switching to "copy char to clipboard" -> "paste from clipboard" thing.

I mean, I imagine that switching layouts just to run a script is a questionable UX, so I'll see whether this hack even works.

Naylepsh commented 1 year ago

Hey, so the fix-special-char-input branch https://github.com/Naylepsh/guild-wars-2-auto-daily-rewards/tree/fix-special-char-input should fix the input issues.

The downside is -- you'll have one additional dependency to install: pyperclip

And here's the above branch extended with partial WQHD support: https://github.com/Naylepsh/guild-wars-2-auto-daily-rewards/tree/feat-resolutions. Note the change in the README -- the initialization of the script changed slightly with this one. Now it's poetry run python -m main --resolution WQHD (or python3 main.py --resolution WQHD in your case)

lukycody commented 1 year ago

Heya, got back to testing. So it actually works if you swap the language to Englisch/US-Keyboard (bottom right if you have another language set, shortcut: WIN+Space). I added the position for the character selection 1st slot. I couldn't really get you the wiggly chest location, as that's completely reliant on the general UI scale and map size.

I will try your solution aswell, but just fyi that it works that way. How do I get the code's version with that branches addition? (sorry, im not really experienced with those git things yet, have tested with a local copy yet) What is it actually doing with pyperclip? Doing the workaround with the clipboard?

However I ran into another problem:

Does the picture detection actually work for you? (Also, why are you not just using pyautogui.click('RemindMeLater.png'))

This is what I found so far. But I think it shouldnt be a problem to take a screenshot with snippet/win+shift+s > paint > save as .png. As its just a cutout of the words anyway. Not sure how to be "pixel-perfect" there.

Naylepsh commented 1 year ago

Assuming you're using git through command line, to test the other branch you simply:

- git pull
- git checkout feat-resolutions

But you might want to save your local changes to your own branch beforehand:

- git checkout -b some-descriptive-branch-name
- git add -A
- git commit -m "some meaningful message"

About pyperclip -- correct, it's just for clipboard handling to enable that previously mentioned workaround.

As for the issues:

  1. I have not personally tested check_for_remind_me_later functionality -- it's something another dev brought. Because the "Remind me later" doesn't appear in my launcher, I didn't even notice when / whether it stopped working. If you can tell me how to have this "Remind me later" show in the launcher, I can take a look into it. (Assuming that it's a priority of some sort, cause I'm guessing you didn't have this reminder popup before?)
  2. Yeah... it's been optimized for my PC, hence the sleep times may require some custom tweaking around on other environments. If you have some specific sleep times that you'd like to use, you can send me the list and I'll create a preset of that.
lukycody commented 1 year ago

Thanks, will try it with git.

remind me later -- it appears at every first attempt of the day per account and then roughly about every 2nd time, when you login to an account, that doesn't have the two-factor-authentication active. If I click it away once, it won't appear atleast on the next attempt but instantly let you launch. It can pop up again on other attempts after though. So it doesnt pop on my main account, but on every alt account.

So yeah, it constantly comes up so its quite important/of priority to get rid off of it, if you wanna automate the logins, as most alt accounts will have that reminder step blocking the launch button.

Would be nice if you could take a look into it. As I said, I already tested quite a bit, and looked up the functionality but couldn't find the issue.

Naylepsh commented 1 year ago

Okay, so I tried it on my end and it looks like it doesn't work for me either. But when I've updated the screenshot it started working, so I'll be committing the new screenshot. However, I'm concerned whether relying on screenshot will work the same in the higher resolutions...

Fyi: I've included the newest image in the most recent version of https://github.com/Naylepsh/guild-wars-2-auto-daily-rewards/tree/feat-resolutions branch

lukycody commented 1 year ago

Ok, will try it with the new screenshot. However that’s weird, because I tried with a new screenshot on my resolution on my own and it didn‘t work. How did you take the sc, via Snipping Tool or windows print shortcut and then insert & save as .png with paint? I wonder if it’s actually this ‘pixel-perfect‘ requirement, that’s not fulfilled if you take a „sloppy“ sc with snipping tool rather than a full resolution fullscreen sc (maybe the methods yield different pixel density)

I also wondered how GW2launcher does it, because it actually skips/prevents this step entirely programmatically (never pops up there). However, it will probably also be a much deeper hook of the process.

Naylepsh commented 1 year ago

I've used snipping tool and then save through paint.

lukycody commented 1 year ago

I tried it with your current branch feat-resolutions (on another computer with FHD) -- it doesn't enter the "@" of the email address (omitted char), always resulting in failed logins. Thus I couldn't test further.

Naylepsh commented 1 year ago

Oof. I should had figured that more characters like that may not be inputted properly on a different keyboard layout...

I've updated the branch -- now the entire input for credentials is handled with the "hack" to hopefully make it layout agnostic. The issue of entering the program's name has also been addressed -- a small delay has been added.

Please let me know whether this update has resolved the aforementioned issues on your end.

lukycody commented 1 year ago

Yep, it works now. Thanks! Thats for FHD though, will test with WQHD aswell. I had a look at the code again and it seems you changed quite a lot about the code structure with the pyperclip/resolution changes (outsourced packages/modules). It was a bit harder to understand for me with those additional modules and restructured code, but I guess its also a bit cleaner. Do you plan to put this into the main branch?

Naylepsh commented 1 year ago

Glad it works now ;)

Yeah, after you confirm WQHD works on your end I do indeed plan to merge it to the main branch.

Do keep in mind that WQHD coordinates are not finished:

    # TODO:
    character=(-1, -1),
    rewards=RewardsCoordinates(x=-1, top_y=-1, bottom_y=-1),

I'm hoping you could provide those :V

lukycody commented 1 year ago

Alright. Yeah sure, I'll do that -- I'm just a bit short on time atm and just on my laptop (FHD). As I said before, those coordinates are not really reliable, as they heavily depend on your ingame setup (e.g. map position up/down and its size). But I'll try to get the coords for the "standard out of the box" setup I guess. Maybe it's also possible to get an average value top- and bottom-y, thats valid for mapsize{min,max}, will check!

Also, to learn, I'd come back to you for some code-understanding questions for this project if that's fine :)

Naylepsh commented 1 year ago

About the "they heavily depend on your ingame setup" -- yeah, these were initially hardcoded to suit my set-up. The whole generate_locations function is supposed to accommodate for some variance in map size, but it's definitely far from perfect. Would greatly appreciate additional coords from your perspective.

And about the "Also, to learn, I'd come back to you for some code-understanding questions for this project if that's fine" -- no problem at all. By all means ask.

lukycody commented 1 year ago

Somehow my launcher seems to have moved a bunch of pixels up (~20px) and right "overnight", so the coordinates of the email/pw field aren't correct anymore (resulting in a click on "reset pw" link instead the pw field for input).

Sure I could manually change them to make it fit - but I'd rather like to understand why it did this and to prevent more launcher wiggles. It seems to always start at the same fixed position (if you drag somewhere else the position isn't saved by the launcher/os). And I didn't change anything from yesterday to today, so I wonder if its position somehow got changed via game files with the minor game updates. Do you have the same issue today on FHD?

Edit: Aaaand it doesn't detect the "remind-me-later.png" anymore. So I think it's got to do with the resolution again, maybe some windows update things? -- But that's weird, as I didn't change anything (resolution is still set to 1920x1080 FHD). Also this implies I'd have to provide a new WQHD .png as it won't work from the FHD .png btw.

Naylepsh commented 1 year ago

Huh, it seems to moved on my end as well... guess that's really due to the update.

And it's back to not detecting the "remind-me-later". Had to re-do the screenshot for it to work again.

Update: the adjustments have been pushed to the feat-resolution branch

lukycody commented 1 year ago

Quick Git question :) So I created my own "personal branch" (named "custom"), just as you've suggested above. Then I've made local changes to the code there (just added and changed minor stuff like a randomizer for the sleeptimers, which I do not want to merge in main in order push it to the repository). So my custom branch should stay local and not be pushed on the repository.

Now I just want to "update" my personal branch with your new changes, so that I have the latest version (changes coords/.png), BUT also keep my personal changes/additions. What is the right way to do it? (internet throws around different methods like fetch and merge, but I don't want to accidently discard my changes by overwriting)

So basically: If I make changes in my custom branch (like changing the set sleeptimers to tailor them to my machine), and there are other changes, that I want to incorporate from the latest feat-resolution into my custom branch -- am I even able to update it without discarding my custom sleep timers? Or will I have to manually resolve conflicts of thos divergent sleeptimer values?

Sorry for blowing this up with general code questions -- but I feel like its really important to understand those git-things to "git gud" ;) (can you even PM at github?)

Naylepsh commented 1 year ago

I don't think you can avoid manually resolving conflicts in this scenario. I'd make a backup branch with your local changes just to be on the safe side, so something like:

git checkout feat-resolution
git pull
git checkout custom
git checkout -b custom-backup
git checkout custom
git merge feat-resolution
... <- the part where you manually resolve conflicts
git add ...
git commit -m ...

This won't magically discard your changes. Worst case scenario you've made a mistake while manually resolving conflicts, but you still have the backup branch to retry.

lukycody commented 1 year ago

Heya @Naylepsh. I know it's been some time and I wanted to get back to you, Sorry for that. Been busy but I didn't forget.

So first off I just wanted to let you know that I've been using the tool for quite some time now and - once correctly set up with the right pause timers for your machine and besides some shenanigans - it worked great so far to login daily with several accounts. I've been exclusively using the tool on my laptop on FHD until now though.

About the shenanigans -- I sometimes encounter the problem, that the script will stop (or rather desync for whatever reason) after the 5min pause window. So it indeed runs the script to the end but the actually correct execution of logging in stopped after the 5min pause (I could verify that by starting GW2 normally and seeing the last saved account (number 21) credentials in the launcher) A thing I noticed is, that in the CLI (where I print out "Account [i] started/finished" for each iteration) at the end of the script (i print out "program finished" at the very end) that a part of the printed out text in the middle of the cmd window is highlighted/marked, so at some point the clicking was desynced and happened with focus on something else than the gw2 launcher window. Couldn't reproduce/see it in real time when it happens yet. It doesn't happen always, but apparently randomly. One reason I could think of is that maybe during the pause windows gives other programs in the background ressource priorities and it somehow breaks the focus on the program.

Okay but that aside theres currently a problem with the copy&paste of account credentials that stops the script from working -- atleast for me. So when it copies the account credentials into the email and password fields, instead of

Happens for me after the update on Tuesday. However, I'm not entirely certain, if thats necessarily connected to the game update or rather something else like the pyperclip dependency/plugin. Can you confirm that problem on your end too? At the moment I have to do the login routine again more manually with gw2launcher 😀

Do keep in mind that WQHD coordinates are not finished

I will look them up when I'm back home at my machine with a WQHD screen. Sorry for me taking so long to do that 😂

lukycody commented 1 year ago

The coordinates of the daily boxes positions in WQHD resolution are as follows (x,y) I assumed a larger UI size with WQHD, because else the UI gets really small, so its unlikely people with use that with this resolution.

with map UI on top:

with map UI on bottom (assuming max. sized map)

lukycody commented 1 year ago

@Naylepsh Hey, sorry to disturb, but do you have any idea how to fix this problem? I had no luck fixing it yet.

Naylepsh commented 1 year ago

Oops, I somehow missed the notifications for the previous messages.

Regarding that >> symbol thingy, yeah, it seems like I've got it on my end as well.

Naylepsh commented 1 year ago

It looks like an issue with the launcher. I tried running the script to output the data to notepad instead of the launcher (you can do the same by replacing "Guild Wars 2" with "notepad":

def _open_gw2_launcher() -> None:
    _open_program("notepad")

) and it pasted the credentials just fine

Naylepsh commented 1 year ago

The dumb solution would be to go back to the naive way of typing the credentials through pressing the keys:

def write_input(xs: str) -> None:
  for x in xs:
     pyautogui.press(x)

but that won't quite work with the non-standard symbols (as we had this issue before, hadn't we)

lukycody commented 1 year ago

Heya -- thanks for replying, I already thought you maybe neglected this small project (which would be sad, it's a nice tool!) 😉 Okay so this behaviour with the launcher is really weird. I wonder what change from the game could result in this. I tested a bit again.

So yeah the "old solution" was problematic, because it assumed an english keyboard and replaced the @ with a *. However, it works for me if I set my machines language/keyboard setting to english (US international) before. Not optimal, but it works I guess.

The bigger problem is that the "clear input function" step somehow now also inputs the symbol » (right at the start of the input field). Not sure which part of the function exactly, testing rn.

def clear_input() -> None:
    pyautogui.keyDown("ctrl")
    pyautogui.press("a")
    pyautogui.keyUp("ctrl")
    pyautogui.press("backspace") #somehow inputs the symbol

Edit: it's the part pyautogui.press("backspace") that inputs that symbol this time. Tried to replace it with "del/delete/clear" but didn't work. Tbh it's not necessary to clear the input as Shift+A selects everthing and the following paste replaces it anyway, but I guess it'd be more concise to actually clear the input before.

Edit2: i removed that line and thought that should do as a working workaround. But the Shift+A "select all" doesn't work properly. It'll just do the doubleclick, which selects the whole word at that cursor location. The Shift+A doesn't seem to activate, so the following input gets placed inside the existing content of the input field

Edit3: the selecting doesn't work for me. can you confirm that? I thought it may be this numlock issue but nope, doesn't work regardless of numlock activated or deactivated...

Naylepsh commented 1 year ago

The selecting didn't work for me as well. It seems like all these "ctrl + something" combinations broke .. I'll take a look if there's some better workaround than `for in range(1000): pyautogui.press("backspace")` lol

Naylepsh commented 1 year ago

So uhh, fun stuff. I tried the very same key inputs (ctrl + a, backspace) with java and it works without any issues there. Ngl, it seems very tempting to migrate

lukycody commented 1 year ago

Well that pyautogui.press("backspace") didn't even work as it put that arrow symbol... so a thousand times wont work aswell 😀 So it seems we don't have a working solution/workaround at the moment huh 🤔

By migrating you mean writing the code anew in Java? Maybe theres also another library for python we could use.

Naylepsh commented 1 year ago

I tried a bunch yesterday but they didn't work at all. I found https://pypi.org/project/pynput/ which seems to do the job (at least when it comes to backspace), so maybe there's hope

Naylepsh commented 1 year ago

Alright, so the feat-resolutions branch should cover the fixes.

lukycody commented 1 year ago

Nice, thank you! Seems to work for me after adding my custom changes (timers) again. Before I pulled I stashed them away and applied them after.

Some more on it though:

Another thing (that already happened before) is that the script stops working after the 5 minute break after 20 accounts, that I built in to prevent the login attempt timeout. Sometimes it runs through normally and continues with #21, sometimes it seems to break focus and end up with the cursor somewhere in the cmd window instead of the game launcher.

Naylepsh commented 1 year ago

I had to renew the "RemindMeLater" picture again on FHD to let it be recognized again.

Yeah, I've noticed that it tends to break on updates, I don't think there's much I can do about that.

I noticed you still had the most of the flow.py/run function still commented out.

Nice catch, completely forgot about it. 😆

Used poetry install again to get the pynput dependency. Getting some message on the imports though

It's just your IDE not using the correct environment.

Sometimes it runs through normally and continues with #21, sometimes it seems to break focus and end up with the cursor somewhere in the cmd window instead of the game launcher

Unfortunately I have no definite clue what's causing it. It could be that every now and then the launching gw2 launcher takes longer than the expected delay and so the login part starts executing on the underlying cmd window instead of the gw2 launcher. Or it could be that the launcher shows in the background instead of foreground (had such a thing happen I think once when I was testing the recent fix) 🤷 You might want to try out extending the sleep time between _open_gw2_launcher and _login and see what happens