thebrumby / HotWalletClaimer

Looking to optimize HOT earnings on Near Protocol, Wave on SUI, Vertus on TON, or other supported apps like Hamster Kombat? This Python script monitors and claims rewards for your favorite Telegram crypto games. It avoids API injections by simulating mouse movements in a real browser.
MIT License
181 stars 48 forks source link

Cleanup docker_data.vhdx #187

Closed farmdigit closed 1 month ago

farmdigit commented 1 month ago

Hi Phil! Thanks for everything you do. Please tell me, the docker file "docker_data.vhdx" is taking up more and more disk space over time. I assume that the accumulation effect is related to cache and other temporary browser files (i think you use selenium). Is there any way to clear this space in the docker file?

thebrumby commented 1 month ago

Hi @farmdigit,

You're right that the docker_data.vhdx file accumulates data over time. This file acts as a virtual hard disk for Docker, storing everything from container data to images, volumes, and more. It's not just related to Selenium caching—though that certainly contributes—but also includes all Docker-managed data.

One approach to managing this growth is to use PM2 log rotation to minimize the retained history, as explained in this previous issue. This method can help reduce the disk space used by limiting the amount of logged data kept over time.

Additionally, the same post describes how you could modify the claimer.py script to switch off Selenium caching, which would prevent these caches from accumulating. Keep in mind that disabling caching will cause every image and script to reload with each claim, potentially slowing down the process.

To clear up space, you can also use Docker commands to remove unused data:

# Remove all unused containers, images, networks, and volumes
docker system prune --all --volumes

Also, their is some suggestion on the internet that you can compact the file, although I never tried it:

# Compact the VHDX file to reclaim space
cd C:\Users\<YourUsername>\AppData\Local\Docker\wsl\data\
Optimize-VHD -Path .\docker_data.vhdx -Mode Full

Finally, you could check to see if there are any unused game accounts by running. If there are idle/unwanted accounts you can delete them:

./launch.sh status

Hope this helps!

Best, Phil

farmdigit commented 1 month ago

Hi @farmdigit,

You're right that the docker_data.vhdx file accumulates data over time. This file acts as a virtual hard disk for Docker, storing everything from container data to images, volumes, and more. It's not just related to Selenium caching—though that certainly contributes—but also includes all Docker-managed data.

One approach to managing this growth is to use PM2 log rotation to minimize the retained history, as explained in this previous issue. This method can help reduce the disk space used by limiting the amount of logged data kept over time.

Additionally, the same post describes how you could modify the claimer.py script to switch off Selenium caching, which would prevent these caches from accumulating. Keep in mind that disabling caching will cause every image and script to reload with each claim, potentially slowing down the process.

To clear up space, you can also use Docker commands to remove unused data:

# Remove all unused containers, images, networks, and volumes
docker system prune --all --volumes

Also, their is some suggestion on the internet that you can compact the file, although I never tried it:

# Compact the VHDX file to reclaim space
cd C:\Users\<YourUsername>\AppData\Local\Docker\wsl\data\
Optimize-VHD -Path .\docker_data.vhdx -Mode Full

Finally, you could check to see if there are any unused game accounts by running. If there are idle/unwanted accounts you can delete them:

./launch.sh status

Hope this helps!

Best, Phil

What if I delete sessions and then recreate them? When deleting all traces of a session are deleted, right? So if a newly created session occupies, for example, 300 MB, over time it grows due to cache and other temporary files up to 2 GB, then if I delete a session that has been running for a long time and recreate a new one will be a partial solution to the problem. Right or not?

thebrumby commented 1 month ago

Hey @farmdigit,

If you're going to remake the sessions, which should save disk space, I've pushed a new update.

If you pull the latest code, and go into any game to edit the settings, there is a new option: enableCache: True

Select the option to update your settings and enter 'n' at the appropriate time to turn it off. Then use ./launch.sh status to delete the accounts, and hopefully when you remake them, it won't cache too many items.

Let me know how you get on.

farmdigit commented 1 month ago

Hey @farmdigit,

If you're going to remake the sessions, which should save disk space, I've pushed a new update.

If you pull the latest code, and go into any game to edit the settings, there is a new option: enableCache: True

Select the option to update your settings and enter 'n' at the appropriate time to turn it off. Then use ./launch.sh status to delete the accounts, and hopefully when you remake them, it won't cache too many items.

Let me know how you get on.

Thanks, I'll try it. I need to:

  1. update the bot ./pull-games.sh
  2. delete current sessions
  3. create new sessions by setting enableCache: false did I understand the sequence of actions correctly?
thebrumby commented 1 month ago

Exactly! Once the setting has been changed to False, it will remain in place for all new or restarted sessions until changed. However, changing the setting to False won't remove previously cached items, which is why you'd need to delete and start with a new session.

farmdigit commented 1 month ago

Exactly! Once the setting has been changed to False, it will remain in place for all new or restarted sessions until changed. However, changing the setting to False won't remove previously cached items, which is why you'd need to delete and start with a new session.

Great! Thanks a lot, I will try it, but it might take a while, I will report back on the result.

farmdigit commented 1 month ago

Exactly! Once the setting has been changed to False, it will remain in place for all new or restarted sessions until changed. However, changing the setting to False won't remove previously cached items, which is why you'd need to delete and start with a new session.

Hi Phil! I deleted all current sessions and now I'm creating new ones and now I'm facing this problem - I don't want to create a new telegram session. What can be the problem? image I tried to enter telegram desktop manually and everything goes fine, but for some reason I can't log in through a bot.

I assume that just the bot does not have time to wait for authorization in telegram and that's why this error appears because some time after the bot gives information about unsuccessful authorization I get a notification about login on a new device in telegram. Maybe increasing the 2FA password waiting interval will solve this problem?

thebrumby commented 1 month ago

Hi,

I'm slightly confused as you said you don't want to create a new Telegram session, but the screenshot is of creating a new Telegram session.

The script currently waits 3 seconds to test for the incorrect password message and then checks if the password entry box is still there.

On my home PC it is <1 second for that box to disappear, but I guess it won't hurt to give some extra seconds, I'll upload that now.

farmdigit commented 1 month ago

I'm slightly confused as you said you don't want to create a new Telegram session, but the screenshot is of creating a new Telegram session.

I want to, but I can't. I misspoke.

The script currently waits 3 seconds to test for the incorrect password message and then checks if the password entry box is still there.

Can this be added as a changeable parameter? To increase the time interval if necessary.

On my home PC it is <1 second for that box to disappear, but I guess it won't hurt to give some extra seconds, I'll upload that now.

on PC too, but it may take longer on the server, it's hard for me to say for sure because I can't see what's going on during authorization, I'm just guessing what the cause is.

thebrumby commented 1 month ago

I want to, but I can't. I misspoke.

OK, that makes sense now. I tested it again today after you raised the issue and it worked OK for me. In the first screenshot, I see it encountered a stale element, do you get that everytime?

Can this be added as a changeable parameter? To increase the time interval if necessary.

If you think it's a problem for your setup, you could edit the script, but there are dozens or hundreds of wait timers throughout the various games, so it would be a big undertaking to modify it, and I'm not really sure that it's the problem anyway. The timer in question is in claimer.py, line 774 increase the number here: (self.driver, 8)

on PC too, but it may take longer on the server, it's hard for me to say for sure because I can't see what's going on during authorization, I'm just guessing what the cause is.

You can always follow the steps in the troubleshooting post, it will let you view the virtual screen at the time of the error. Screenshots are prefixed with the step number, so just look for the ones beginning "09"

farmdigit commented 1 month ago

If you pull the latest code, and go into any game to edit the settings, there is a new option: enableCache: True

Unfortunately, this setting seems to cause bugs when adding a new session - for example, it is impossible to add a new spell wallet session in principle - an error constantly pops up as on the screenshot. Perhaps the problem with login to telegram, which I wrote above, is caused by the same reason in fact. image

thebrumby commented 1 month ago

enableCache: True

Unfortunately, this setting seems to cause bugs when adding a new session - for example, it is impossible to add a new spell wallet session in principle - an error constantly pops up as on the screenshot.

Hi,

The setting you're referring to simply controls whether Chrome caches items like images, CSS, or JavaScript files. I’ve tested the session creation with a game involving a seed phrase, and it works fine whether the cache is enabled or disabled.

Today, I re-tested Spell by starting with a fresh Telegram session. I logged in using a phone number and one-time password on an account with 2-factor authentication, and the script ran without any issues.

As previously mentioned, please follow the troubleshooting guide to enable verbose level 3. This will help identify which step is failing. Additionally, enabling the debugging setting will allow you to check the screenshots to see exactly what happens during the claims process.

Thanks.

farmdigit commented 1 month ago

to enable verbose level 3

sure, here's a screenshot with the settings verbose 3 image

at the same time with the caching setting enabled, everything works and new sessions are registered fine.

thebrumby commented 1 month ago

OK, so if I understand correctly:

The scenario I didn't test is:

I think the solution to this issue is probably:

        if not self.settings.get("enableCache", True) and step_int >= 100:
            chrome_options.add_argument("--disable-application-cache")

So basically, the only time caching is disabled is if enableCache is False, and we've finished the Telegram setup and moved on to the main claim phase (steps > 100).

Hopefully, this will resolve the issue :)

farmdigit commented 1 month ago

If you think it's a problem for your setup, you could edit the script, but there are dozens or hundreds of wait timers throughout the various games, so it would be a big undertaking to modify it, and I'm not really sure that it's the problem anyway. The timer in question is in claimer.py, line 774 increase the number here: (self.driver, 8)

I am still inclined that the reason is that due to disabling caching the loading time increases and this is the cause of the error. please tell me how to realize it "The timer in question is in claimer.py, line 774 increase the number here: (self.driver, 8)" through docker? I don't understand.

Another question is why in Poketfi the interval between claimes is 4 hours, because there are 6 hours. How can it be corrected?

thebrumby commented 1 month ago

Hi,

The change I suggested to line 774 was intended to address the time-out error you encountered during 2-factor authentication (2FA). In your follow-up screenshot, you showed an error during the creation of a new game session with a saved Telegram account. I understood that this error only occurred when enableCaching was set to False, so I pushed a small fix to force caching during these steps. Now, we're back to discussing line 774 again. If you're using a saved session, it wouldn't trigger the 2FA check at line 774 since the session has already passed that point.

To move forward, I recommend ensuring your code is up-to-date, deleting the saved Telegram account, and starting fresh. If the issue persists, refer to your screenshots to identify the problem, as outlined in the Troubleshooting guide. If the issue is still related to 2FA, you could temporarily disable 2FA in your Telegram account, authenticate the session, and then re-enable it afterward. If you prefer to keep 2FA active, which I use without any problems, the screenshots should help you identify what's going wrong.

Regarding PocketFi, it burns your pending tokens if they aren't claimed within 6 hours. The 4-hour interval was chosen under the assumption that if users had a random offset timer, it would likely be less than 2 hours, ensuring the claim is made before the timer expires and the coins are lost. If you prefer a different strategy, you'll need to edit the configuration yourself using a text editor like Nano.

farmdigit commented 1 month ago

Hi,

The change I suggested to line 774 was intended to address the time-out error you encountered during 2-factor authentication (2FA). In your follow-up screenshot, you showed an error during the creation of a new game session with a saved Telegram account. I understood that this error only occurred when enableCaching was set to False, so I pushed a small fix to force caching during these steps. Now, we're back to discussing line 774 again. If you're using a saved session, it wouldn't trigger the 2FA check at line 774 since the session has already passed that point.

To move forward, I recommend ensuring your code is up-to-date, deleting the saved Telegram account, and starting fresh. If the issue persists, refer to your screenshots to identify the problem, as outlined in the Troubleshooting guide. If the issue is still related to 2FA, you could temporarily disable 2FA in your Telegram account, authenticate the session, and then re-enable it afterward. If you prefer to keep 2FA active, which I use without any problems, the screenshots should help you identify what's going wrong.

Regarding PocketFi, it burns your pending tokens if they aren't claimed within 6 hours. The 4-hour interval was chosen under the assumption that if users had a random offset timer, it would likely be less than 2 hours, ensuring the claim is made before the timer expires and the coins are lost. If you prefer a different strategy, you'll need to edit the configuration yourself using a text editor like Nano.

Hi Phil! I have 2 more questions when creating sessions:

  1. Are these settings shown on the screenshot for each session individual? Or when changing settings when creating a new session, all settings are changed?
  2. if the settings for each session are individual can they be changed for an already created session? or is the only way to change them is to delete the old session and start a new one? image

how and where do I make corrections to the code in the docker?

    if not self.settings.get("enableCache", True) and step_int >= 100:
        chrome_options.add_argument("--disable-application-cache")
thebrumby commented 1 month ago

Hi,

The settings are grouped per folder. On docker if you want different groups of games to have different settings, you can spin up multiple containers. In the stand-alone install, you can create different folders with individual settings.

Restarting a session in PM2 applies the latest settings without the need to recreate a session.

farmdigit commented 1 month ago

Hi,

The settings are grouped per folder. On docker if you want different groups of games to have different settings, you can spin up multiple containers. In the stand-alone install, you can create different folders with individual settings.

Restarting a session in PM2 applies the latest settings without the need to recreate a session. Hi! Can you please tell me why this error is shown? It crashes when creating a new session in different places - when creating a new telegram session, when creating a new bot session after entering a seed-phrase. What does it mean and what is the reason? image

thebrumby commented 1 month ago

Can you please tell me why this error is shown? It crashes when creating a new session in different places - when creating a new telegram session, when creating a new bot session after entering a seed-phrase. What does it mean and what is the reason?

I copied the wrong tag from the self.increase_step() method. I've corrected it and pushed it fix to the repository, kindly update your codebase as described above and try again.

farmdigit commented 1 month ago

Can you please tell me why this error is shown? It crashes when creating a new session in different places - when creating a new telegram session, when creating a new bot session after entering a seed-phrase. What does it mean and what is the reason?

I copied the wrong tag from the self.increase_step() method. I've corrected it and pushed it fix to the repository, kindly update your codebase as described above and try again.

update the bot ./pull-games.sh Right?

thebrumby commented 1 month ago

On Docker, yes. Commands are here.

farmdigit commented 1 month ago

I copied the wrong tag from the self.increase_step() method. I've corrected it and pushed it fix to the repository, kindly update your codebase as described above and try again.

errors still appear when creating a hot session with caching turned off. image

And even when after several attempts I somehow manage to start a session, this happens in the log endlessly. The thread is constantly busy image

Maybe we need to fix something there, too?

thebrumby commented 1 month ago

Hi,

I just deleted one of my accounts and set it up again with caching disabled.

Setup steps

image

Main claim steps

image

As mentioned 3 times already in the answer above if you enable the debugging setting, and check the screenshots against the step that fails to find out what's happening - here is an example of one of my screenshots from the session above, as you can see, there are no issues even with caching disabled.

In-game debugging screenshots

image

The troubleshooting guide covers all of this in detail!

Troubleshooting instructions

image

thebrumby commented 1 month ago

One other thing to consider, if you have been using the same Docker container, and there now seem to be a lot of errors that I can't recreate, you can always delete the container and start again with a new one - so it would be like a total fresh install.

Docker Instructions

Remove the current container

image

Start again

image

farmdigit commented 1 month ago

Hi,

I just deleted one of my accounts and set it up again with caching disabled.

Setup steps

image

Main claim steps

image

As mentioned 3 times already in the answer above if you enable the debugging setting, and check the screenshots against the step that fails to find out what's happening - here is an example of one of my screenshots from the session above, as you can see, there are no issues even with caching disabled.

In-game debugging screenshots

image

The troubleshooting guide covers all of this in detail!

Troubleshooting instructions

image

Hi Phil! I have a constantly busy thread right now because the spell claims in circles at 1 hour intervals. So i can't do anything :) what's the reason? I assume that the intention was to check every hour for a daily quiz and today this quiz was quite late, but even on those accounts where it will pass the next claim is still set after 1 hour :) image & log image

I assume that when you edit interval to check on daily quiz you forgot to return normal value interval between claimes after successful passage of quiz. Right?

I'm guessing it's this piece of code (string 134-144 in spell.py):

Calculate remaining time in hours

    try:
        hourly_profit = float(self.get_profit_hour(True))
        xpath = "//div[@id='slider-root-:r5:']/following-sibling::p"
        elapsed = self.monitor_element(xpath, 10, "Get the timer bar")
        current, max_value = map(float, elapsed.split('/'))
        remaining_time_hours = (max_value - current) / hourly_profit
        theoretical_timer = remaining_time_hours * 60
    except Exception as e:
        print(f"An error occurred: {e} - Assigning 1 hour timer")
        theoretical_timer = 60

for some reason the exception is triggered even when the daily quiz is successful

thebrumby commented 1 month ago

Hey @farmdigit,

You were right with your revised comment—it wasn’t linked to the new bonus claim feature we rolled out. The issue actually stemmed from a broken XPath used in calculating the remaining wait time until the pot is full.

I get a lot of support tickets due to temporary errors when reading numbers or text from the game that's later used in a calculation, so I put a fallback mechanism in place to catch these errors and default to a 60-minute retry timer.

I was quite proud of my original solution to calculate the remaining time until the pot was full:

image
  1. Retrieve the pot’s total mana capacity from the GUI (e.g., 0.5 in the screenshot above).
  2. Determine the current mana level in the pot (e.g., 0.025).
  3. Calculate the remaining mana needed by subtracting the current amount from the total capacity.
  4. Obtain the fill rate (e.g., 0.120 per hour) and calculate the remaining time by dividing the remaining mana by the fill rate.

Normally, I prioritize using XPaths based on straightforward text labels or similar identifiers. However, when it came to checking the progress bar for the remaining time, I couldn’t find anything nearby to use. This led me to rely on this rather clunky solution:

xpath = "//div[@id='slider-root-:r5:']/following-sibling::p"

The problem with this type of locator—dependent on obscure IDs or class references—is that any changes by the game developers can break our script, requiring constant updates. The issue you noticed was a result of such a change, where the script couldn’t locate the necessary elements in steps 1 and 2, causing it to default to the fallback 60-minute timer every time the game ran.

xpath = "//p[contains(., '/')]"

What I overlooked at the time, when coming up with the original XPath, was that the remaining timer includes a forward slash between the two numbers, which is unique to that element. By switching to this approach, we now have a more robust solution that won’t be affected by changes to the div IDs.

Cheers, Phil

farmdigit commented 1 month ago

Hey @farmdigit,

You were right with your revised comment—it wasn’t linked to the new bonus claim feature we rolled out. The issue actually stemmed from a broken XPath used in calculating the remaining wait time until the pot is full.

I get a lot of support tickets due to temporary errors when reading numbers or text from the game that's later used in a calculation, so I put a fallback mechanism in place to catch these errors and default to a 60-minute retry timer.

I was quite proud of my original solution to calculate the remaining time until the pot was full:

image
  1. Retrieve the pot’s total mana capacity from the GUI (e.g., 0.5 in the screenshot above).
  2. Determine the current mana level in the pot (e.g., 0.025).
  3. Calculate the remaining mana needed by subtracting the current amount from the total capacity.
  4. Obtain the fill rate (e.g., 0.120 per hour) and calculate the remaining time by dividing the remaining mana by the fill rate.

Normally, I prioritize using XPaths based on straightforward text labels or similar identifiers. However, when it came to checking the progress bar for the remaining time, I couldn’t find anything nearby to use. This led me to rely on this rather clunky solution:

xpath = "//div[@id='slider-root-:r5:']/following-sibling::p"

The problem with this type of locator—dependent on obscure IDs or class references—is that any changes by the game developers can break our script, requiring constant updates. The issue you noticed was a result of such a change, where the script couldn’t locate the necessary elements in steps 1 and 2, causing it to default to the fallback 60-minute timer every time the game ran.

xpath = "//p[contains(., '/')]"

What I overlooked at the time, when coming up the XPath, was that the remaining timer includes a forward slash between the two numbers, which is unique to that element. By switching to this approach, we now have a more robust solution that won’t be affected by changes to the div IDs.

Cheers, Phil

Thanks a lot Phil! But I updated the bot and the problem still remains. image

thebrumby commented 1 month ago

You said the problem was it doing it hourly, and now I changed and it's doing it 3 or 4 times a day it's still a problem?

farmdigit commented 1 month ago

You said the problem was it doing it hourly, and now I changed and it's doing it 3 or 4 times a day it's still a problem?

I misunderstood. I thought that problem was solved.

By switching to this approach, we now have a more robust solution that won’t be affected by changes to the div IDs.

So the only way to do anything is to increase the interval? But then the bot will always collect at a given interval.

thebrumby commented 1 month ago

Virtually every script has the same format:

  1. The script calculates the time until the pot is full.
  2. The user has the option to set an offset to modify the timer - example are the the main readme
    • the main exception is games that don't allow to claim before the timer elapses, any random offset below zero is changed to 0
    • and Pocket Fi which will burn the tokens at 6 hours, so it claims at 4 hours +/- the offset
  3. The script will claim the available coins once the wait timer elapses
  4. Any additional routines such as daily reward, level upgrades. auto-staking, solve the puzzle etc are checked when the claim timer elapses.