AdriaGual / marvel-snap-bot

A computer vision bot made with OpenCV and ADB.
92 stars 24 forks source link

Images not included #1

Closed jimbob4842 closed 1 year ago

jimbob4842 commented 1 year ago

The bot cannot run without the images folder which is not included in this repository.

AdriaGual commented 1 year ago

Due to the TOS of Marvel Snap, I'm not allowed to share this content. But here's the list of the needed files and the folder structure if you wanna build it yourself!

All the images are in .png

. ├── data # Images of the cards you wanna play │ └── antman │ └── .pngs # Don't need to have any name, but I highly recommend add at least 2-3 images or more └── images ├── next_button ├── play_button ├── mana │ ├── 0_mana │ ├── 1_mana │ ├── 2_mana │ └── ... ├── turns │ ├── 1 │ ├── 2 │ ├── 3 │ ├── ... │ ├── next_button │ ├── play_button │ ├── collect_rewards │ └── final_turn └── fields ├── kyln ├── fisk_tower ├── kamar_taj └── ...

jimbob4842 commented 1 year ago

I got it to click the Play button to find a game but I can't get it to detect the turn

The log is full of "[get_turn] Time elapsed: 0:00:00.024179"

Can you tell me what the image should be?

Fofo95 commented 1 year ago

I got it to click the Play button to find a game but I can't get it to detect the turn

The log is full of "[get_turn] Time elapsed: 0:00:00.024179"

Can you tell me what the image should be?

So you created files using the folder structure in the second comment? Do you need to add .png manually first?

AdriaGual commented 1 year ago

@jimbob4842 To detect the turn you need to fill the folder /images/turns with the images of each turn (1/6, 2/6, 3/,6 and so on), you can see it when playing a match, on the bottom right of the screen.

@Fofo95 As mentioned before, due to the TOS of Marvel Snap, I'm not allowed to share the pictures so yes, I recommend you to grab them and yes, they must be in .png format, I just used the default 'Snipping Tool', nothing fancy there.

Hope it helps!

jimbob4842 commented 1 year ago

@jimbob4842 To detect the turn you need to fill the folder /images/turns with the images of each turn (1/6, 2/6, 3/,6 and so on), you can see it when playing a match, on the bottom right of the screen.

@Fofo95 As mentioned before, due to the TOS of Marvel Snap, I'm not allowed to share the pictures so yes, I recommend you to grab them and yes, they must be in .png format, I just used the default 'Snipping Tool', nothing fancy there.

Hope it helps!

What's the resolution of your emulator? I think that might be why it's not working.

AdriaGual commented 1 year ago

1600 x 900 Landscape, Graphics OpenGL + Performance, will update the README.md when I have more time

jimbob4842 commented 1 year ago

I was able to get it to detect the turn by cropping it from the screenshots ADB took. It must've had something to do with the tolerance caused by the difference in resolution when I used snipping tool.

azrulshah commented 1 year ago
Traceback (most recent call last):
  File "C:\Users\marvel-snap-bot\start.py", line 14, in <module>
    android_connection.connect()
  File "C:\Users\marvel-snap-bot\utils\android_connection.py", line 9, in connect
    os.chdir(config.adb_path)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'D:\\JocsPC\\Nox\\bin'

my error can someone help?

Fofo95 commented 1 year ago
Traceback (most recent call last):
  File "C:\Users\marvel-snap-bot\start.py", line 14, in <module>
    android_connection.connect()
  File "C:\Users\marvel-snap-bot\utils\android_connection.py", line 9, in connect
    os.chdir(config.adb_path)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'D:\\JocsPC\\Nox\\bin'

my error can someone help?

Try changing the paths to match your paths in config.py file

Fofo95 commented 1 year ago

I was able to get it to detect the turn by cropping it from the screenshots ADB took. It must've had something to do with the tolerance caused by the difference in resolution when I used snipping tool.

Is there a way you could help me to make it work?

azrulshah commented 1 year ago
Traceback (most recent call last):
  File "C:\Users\marvel-snap-bot\start.py", line 14, in <module>
    android_connection.connect()
  File "C:\Users\marvel-snap-bot\utils\android_connection.py", line 9, in connect
    os.chdir(config.adb_path)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'D:\\JocsPC\\Nox\\bin'

my error can someone help?

Try changing the paths to match your paths in config.py file

still cant, can i see an example?

AdriaGual commented 1 year ago

@Fofo95 I've updated the README.md, there are 2 functions in the info.py with a parameter you can turn True and it'll show you exactly the images with the resolution you are looking for (get_my_hand_cards and get_fields). You can also take a look at how cv2 makes the image and replicate it to get the mana and the turns.

@azrulshah You have to open the config.py file and edit the paths, the error you is because the path to the ADB is not set correctly, make sure you have ADB and check the folder where you can execute it from the cmd (just do an 'adb devices' to check if you have it).

Hope it helps!

Fofo95 commented 1 year ago

@Fofo95 I've updated the README.md, there are 2 functions in the info.py with a parameter you can turn True and it'll show you exactly the images with the resolution you are looking for (get_my_hand_cards and get_fields). You can also take a look at how cv2 makes the image and replicate it to get the mana and the turns.

@azrulshah You have to open the config.py file and edit the paths, the error you is because the path to the ADB is not set correctly, make sure you have ADB and check the folder where you can execute it from the cmd (just do an 'adb devices' to check if you have it).

Hope it helps!

Thanks, I'll try and let you know!

jimbob4842 commented 1 year ago

Instead of using images for every field you can read the text from the screenshot

import pytesseract
import cv2
import numpy as np

image = cv2.imread('1.png')
pytesseract.pytesseract.tesseract_cmd = "C:\\Program Files\\Tesseract-OCR\\tesseract.exe"

bgr = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
lower_color = np.array([200, 200, 200])
upper_color = np.array([255, 255, 255])
mask = cv2.inRange(bgr, lower_color, upper_color)
image[mask == 0] = [0, 0, 0]

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
_, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

text = pytesseract.image_to_string(thresh, lang='eng')

print(text)

With some tinkering it can probably be used to read most text including the current turn and mana.

azrulshah commented 1 year ago

@Fofo95 I've updated the README.md, there are 2 functions in the info.py with a parameter you can turn True and it'll show you exactly the images with the resolution you are looking for (get_my_hand_cards and get_fields). You can also take a look at how cv2 makes the image and replicate it to get the mana and the turns.

@azrulshah You have to open the config.py file and edit the paths, the error you is because the path to the ADB is not set correctly, make sure you have ADB and check the folder where you can execute it from the cmd (just do an 'adb devices' to check if you have it).

Hope it helps!

C:\Users\Azrul\marvel-snap-bot>python start.py Traceback (most recent call last): File "C:\Users\Azrul\marvel-snap-bot\start.py", line 1, in from utils import android_connection, global_utils File "C:\Users\Azrul\marvel-snap-bot\utils\android_connection.py", line 4, in import config File "C:\Users\Azrul\marvel-snap-bot\config.py", line 2 project_path = "C:\Users\Azrul\marvel-snap-bot" ^ SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

jimbob4842 commented 1 year ago

@Fofo95 I've updated the README.md, there are 2 functions in the info.py with a parameter you can turn True and it'll show you exactly the images with the resolution you are looking for (get_my_hand_cards and get_fields). You can also take a look at how cv2 makes the image and replicate it to get the mana and the turns. @azrulshah You have to open the config.py file and edit the paths, the error you is because the path to the ADB is not set correctly, make sure you have ADB and check the folder where you can execute it from the cmd (just do an 'adb devices' to check if you have it). Hope it helps!

C:\Users\Azrul\marvel-snap-bot>python start.py Traceback (most recent call last): File "C:\Users\Azrul\marvel-snap-bot\start.py", line 1, in from utils import android_connection, global_utils File "C:\Users\Azrul\marvel-snap-bot\utils\android_connection.py", line 4, in import config File "C:\Users\Azrul\marvel-snap-bot\config.py", line 2 project_path = "C:\Users\Azrul\marvel-snap-bot" ^ SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

Put \\ instead of \

azrulshah commented 1 year ago

Traceback (most recent call last): File "C:\Users\Azrul\marvel-snap-bot\start.py", line 1, in from utils import android_connection, global_utils File "C:\Users\Azrul\marvel-snap-bot\utils\global_utils.py", line 3, in import cv2 ModuleNotFoundError: No module named 'cv2'

sorry if i post here, any way i can contact developer?

jimbob4842 commented 1 year ago

Traceback (most recent call last): File "C:\Users\Azrul\marvel-snap-bot\start.py", line 1, in from utils import android_connection, global_utils File "C:\Users\Azrul\marvel-snap-bot\utils\global_utils.py", line 3, in import cv2 ModuleNotFoundError: No module named 'cv2'

sorry if i post here, any way i can contact developer?

pip install -r requirements.txt

This is the best way to contact them.

azrulshah commented 1 year ago

Traceback (most recent call last): File "C:\Users\Azrul\marvel-snap-bot\start.py", line 1, in from utils import android_connection, global_utils File "C:\Users\Azrul\marvel-snap-bot\utils\global_utils.py", line 3, in import cv2 ModuleNotFoundError: No module named 'cv2' sorry if i post here, any way i can contact developer?

pip install -r requirements.txt

This is the best way to contact them.

ERROR: Cannot install -r requirements.txt (line 2) and numpy==1.21.0 because these package versions have conflicting dependencies.

The conflict is caused by: The user requested numpy==1.21.0 opencv-python 4.6.0.66 depends on numpy>=1.21.2; python_version >= "3.10"

To fix this you could try to:

  1. loosen the range of package versions you've specified
  2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

or how can i contact u, so that i dont have to spam here, discord maybe

MarcinMiko commented 1 year ago

I changed the numpy version to 1.22.0 and worked for me

cheryiit commented 1 year ago

i dont have images what can i do?

MarcinMiko commented 1 year ago

@cheryiit From what I read you need to make them yourself as otherwise it's against the TOS of Marvel snap. @AdriaGual I do have a question as I'm just starting my journey with Python and I managed to run the code in Visual studio (doesn't matter where I hope) It's connected but first thing I'm getting is already the [Get_turn] Connection result: already connected to 127.0.0.1:5555

[get_turn] Time elapsed: 0:00:00.014426 [get_turn] Time elapsed: 0:00:00.015442 I have the game open on the main screen why is it not trying to first click the go_to_play_button or play_button Update while writing (I tried to crop the screenshot in tmp folder and that started to kinda work) but now it's just clicking my deck selector instead of the Play button. (I made the images of each turn in the turns folder)- I assume will have to do that for everything from adb screenshots. Am I doing something out of order?

azrulshah commented 1 year ago

I changed the numpy version to 1.22.0 and worked for me

still same, any other things i have to check?

cheryiit commented 1 year ago

anyone share with hash64 link pls?

AdriaGual commented 1 year ago

@azrulshah cv2 is the python package for opencv, to install it check here: opencv As @jimbob4842 said, the best way to install the dependencies is using the requirements.txt to install the dependencies on the version needed (the version is important, opencv must be at 4.6.0.66 and numpy at 1.21.0, you can do a pip freeze to check all the python packages you have installed on your system). If you have different versions, try uninstalling them and installing the mentioned versions.

As @MarcinMiko said, Marvel Snap has a strict TOS policy, especially against this type of scripts (nothing new to be honest). I thought that the best way was to let each user take their screenshots, since it would be harder for them to see this repo, but since it's generating a bit of trouble, I have uploaded a set of the haystack images needed to run the bot 🤥, you can pull the data and images folders and try running the script (let's hope they don't put their eyes here haha). Just to clarify, the cards in Marvel Snap resize depending on the number of cards on hand (from 1 to 7) and the position of them (the ones on the sides are a bit rotated left or right), so many times you need to have multiple resolution images of each card.

ADD template images +add spectrum values+threshold

I'm also working on ways to improve this, like masking the whole image so I can get rid of the unnecessary information (background, buttons I already know where they are, etc..) or using pytesseract to get the points from the player and the opponent so we can make better plays!

Hope it helps!

azrulshah commented 1 year ago

@azrulshah cv2 is the python package for opencv, to install it check here: opencv

it works tq boss, and now can play the bot, but always stuck on turn 3, another thing want to ask, if let say want to play angela, but in my game, angela is using different car variant, can it still be detect? and what if the heroes is not in the image folder? let say galactus, i have to manually add?

plus it seems not detect fields correctly, wish i can help

AdriaGual commented 1 year ago

@azrulshah hmm the stuck on turn 3 problem I can't replicate it, I've added a bunch more of turn 3 screenshots, pull when you can.

If it's a variant card, I'm sad but you'll have to do it (I don't have it yet xD), you can enable the my_hands_card parameter in the get_info function in order to see the hand, and then crop a bunch of angela images (would be nice to have 3-4 on different positions and different card size). image

With the fields what happens is that I don't have all of them, to add a field, simply enable the active_fields parameter on the function, check the fields that will appear as screenshots and crop the title (just 1 is fine). Then save it in the fields folder with the corresponding name and, ⚠️IMPORTANT⚠️ you need to add that field to the field_list.py, for now, with the default_config it's enough, this has been made to prioritize the fields to where the scripts wants to play a card.

Here you can check how a field is created: COMMIT

Also, since I feel that changing this parameter everytime someone wants to check an image maybe is kinda clunky, I've created 3 separated scripts that do this! print_fields.py -> Shows in real time screenshots of the fields. print_hand_cards.py -> Shows in real time a screenshot of the hands cards zone. print_turn.py -> Shows in real time a screenshot of the turn,

Hope this helps!

azrulshah commented 1 year ago

yes, all done and working, but i want to ask, is the bot are intelligent? i mean if put angela first then later fill up other card there to get points, or the card is random play? any function of the field play? i mean did the bot detects altar of death or bar sinister power?

Fofo95 commented 1 year ago

I tried to use the bot just to click on End turn, collect rewards and click on play again since I have Agatha but I couldn't make these 3 action work, just the end turn is working, any idea?

AdriaGual commented 1 year ago

@azrulshah This is the next step, now each card has an average_cp calculated hardcoded (check the cp_list.py) and each field has a priority (fields that have positive effects have more priority than others). There's a function called play_cards(play_info, last_move) in the hand_cards.py where it gets the card with the highest value and puts it on the most priority field. But you can override this behaviour with custom logic (example: 'if field is empty and i have angela in hand and i have 2 or more mana' -> play angela.

@Fofo95 Check emulator settings, resolution and python packages version.

Hope it helps!

dkcho679 commented 1 year ago

Traceback (most recent call last): File "C:\Users\Azrul\marvel-snap-bot\start.py", line 1, in from utils import android_connection, global_utils File "C:\Users\Azrul\marvel-snap-bot\utils\global_utils.py", line 3, in import cv2 ModuleNotFoundError: No module named 'cv2' sorry if i post here, any way i can contact developer?

pip install -r requirements.txt This is the best way to contact them.

ERROR: Cannot install -r requirements.txt (line 2) and numpy==1.21.0 because these package versions have conflicting dependencies.

The conflict is caused by: The user requested numpy==1.21.0 opencv-python 4.6.0.66 depends on numpy>=1.21.2; python_version >= "3.10"

To fix this you could try to:

1. loosen the range of package versions you've specified

2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

or how can i contact u, so that i dont have to spam here, discord maybe

where did you get the ADB path? Can't find

dkcho679 commented 1 year ago

The problem with the numpy and open cv with python 3.10 and 3.11 need 3.9

azrulshah commented 1 year ago

anyone want to share other images for heroes? i cant get this to work

https://user-images.githubusercontent.com/25803231/211024589-c4545e4b-523c-40f0-b1c1-3302bf1ed83b.png