dolphin-selenium getting inactive after a while

BersXx commented 5 months ago

For some reason from time to time I have to open anty dolphin software and refresh the browsers tab for the API to work again, can you give me your advice on this DedInc? thank you, love your work!

BersXx commented 5 months ago

I think I found out what my problem is, for some reason api.delete_profiles([profile_id]) stop deleting profiles after like 30mins from running the code, not sure why tho, creating still working but not deleting for some reason

BersXx commented 5 months ago

Hey, DedInc, I think there is an issue with the delete profiles API thing, after deleting more than 30 profiles they all came back once again after a while, not sure why

This is related to my problem above, this is the real problem, I am still looking into it

DedInc commented 5 months ago

I tried debugging this

    def delete_profiles(self, ids):
        r = self.s.delete(
            'https://dolphin-anty-api.com/browser_profiles?forceDelete=1', json={'ids': ids})
        try:
            print(r.text) # debug line
            return r.json()
        except:
            raise RuntimeError(r.text)

and I got this log (86 times in a row):

{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}
{"success":true}

Try debugging this too, maybe you will understand what the problem is, or the problem is something else (not in the API itself).

BersXx commented 5 months ago

Thank you for your response DedInc, i am testing your code atm, for the mean while can you test mine? I keep getting this error

This is the error:

Exception in thread Thread-5 (run_browser_automation):
Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1052, in _bootstrap_inner
    self.run()
  File "C:\Users\User\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 989, in run
    self._target(*self._args, **self._kwargs)
  File "z:\Test code.py", line 19, in run_browser_automation
    port = response['automation']['port']
           ~~~~~~~~^^^^^^^^^^^^^^
KeyError: 'automation'

This is my code, I am using the example code you gave us which is great, I have just added threading to it, after running this code for like 30-60min I keep getting that error and sometimes when I start the code

import random
import threading
import selenium_dolphin as dolphin
from selenium_dolphin import DolphinAPI, STABLE_CHROME_VERSION
from selenium.webdriver.chrome.options import Options

def run_browser_automation(api_key):
    api = DolphinAPI(api_key='You API Key')

    while True:
        fingerprint = []
        while not fingerprint:
            fingerprint = api.generate_fingerprint(platform='windows', browser_version=f'{random.randint(114, STABLE_CHROME_VERSION)}')

        data = api.fingerprint_to_profile(name='my superprofile', fingerprint=fingerprint)
        profile_id = api.create_profile(data)['browserProfileId']

        response = dolphin.run_profile(profile_id)
        port = response['automation']['port']

        options = Options()
        options.add_argument("--start-maximized")

        driver = dolphin.get_driver(options=options, port=port)
        driver.get('https://google.com/')
        print(driver.title)
        driver.quit()

        dolphin.close_profile(profile_id)
        api.delete_profiles([profile_id])

# Threads list
threads = []

# Creating 5 threads
for i in range(5):
    thread = threading.Thread(target=run_browser_automation, args=('your_api_key_here',))
    threads.append(thread)
    thread.start()

# Joining all threads
for thread in threads:
    thread.join()

DedInc commented 5 months ago

this error

my working way with threads:

import asyncio
import random
from loguru import logger
from threading import Lock
import selenium_dolphin as dolphin
from selenium_dolphin import DolphinAPI, STABLE_CHROME_VERSION
from concurrent.futures import ThreadPoolExecutor
from string import ascii_letters, digits
from config import dolphin_key
import nltk

file_lock = Lock()

try:
    words = nltk.corpus.words.words()
except:
    nltk.download('words')
    words = nltk.corpus.words.words()

api = DolphinAPI(api_key=dolphin_key)

logger.info('Getting list of profiles')
response = api.get_profiles()
if response['data']:
    for profile in response['data']:
        profile_id = profile['id']
        if profile_id:
            logger.info(f'Deleting profile {profile_id}')
            api.delete_profiles([profile_id])

valid_proxies = []
with open('proxies.txt', 'r') as f:
    valid_proxies = f.read().splitlines()

def process_profile(i, proxy):
    try:        
        logger.info(f'Profile {i+1} | Generating fingerprint')
        fingerprint = []
        while not fingerprint:
            fingerprint = api.generate_fingerprint(platform='windows', browser_version=f'{random.randint(114, STABLE_CHROME_VERSION)}')

        username = f'{random.choice(words)}{random.randint(1000, 9999)}'
        password = ''.join(random.choices(ascii_letters + digits, k=random.randint(8, 12)))

        logger.info(f'Profile {i+1} | Creating profile')
        data = api.fingerprint_to_profile(name=f'profile{i+1}', fingerprint=fingerprint)

        if '@' in proxy:
            dem = proxy.split('@')
            address = dem[1]
            ip, port = address.split(':')
            creds = dem[0]
            login, password = creds.split(':')
        else:
            ip, port = proxy.split(':')
            login = None
            password = None

        proxy_dict = {'proxy': {
            'name': f'socks5://{proxy}',
            'host': ip,
            'port': port,
            'type': 'socks5'
        }}

        if login and password:
            proxy_dict['proxy'].update({
                'login': login,
                'password': password
        })

        data.update(proxy_dict)

        profile_id = api.create_profile(data)['browserProfileId']

        logger.info(f'Profile {i+1} | Launching profile')
        response = dolphin.run_profile(profile_id, True)
        port = response['automation']['port']
        ws_endpoint = response['automation']['wsEndpoint']

        from mailgw import MailGW
        import re
        async def main(profile_num, username, password):
            clz = False
            try:
                browser = await dolphin.get_browser(ws_endpoint, port)
                pages = await browser.pages()
                page = pages[0]

                logger.info(f'Profile {profile_num} | Going to registration page')
                await page.goto('https://ru3.bongacams.com/continue-18-plus', waitUntil='networkidle0')
                try:
                    logo_link_exists = await page.waitForSelector('.logo_link')
                except:
                    logo_link_exists = None

                if logo_link_exists:
                    await page.goto('https://ru3.bongacams.com/members/join')
                else:
                    logger.error(f'Profile {profile_num} | Failed to go to registration page')
                    raise Exception('Failed to navigate to registration page')

                logger.info(f'Profile {profile_num} | Filling out registration form')
                username_field = await page.waitForSelector('#user_member_username')
                await username_field.type(username)

                password_field = await page.waitForSelector('#user_member_password')  
                await password_field.type(password)

                checkbox_field = await page.waitForSelector('#user_member_terms_of_use')
                await checkbox_field.click()

                await asyncio.sleep(0.3)

                submit_button = await page.waitForSelector('.join_submit')
                await submit_button.click()

                account_card = await page.waitForSelector(f'.name[title="{username}"]')
                await account_card.click()

                try:
                    mail_field = await page.waitForSelector('input[name="email_popup"]')
                except:
                    logger.error(f'Profile {profile_num} | Failed to find email input field')
                    raise Exception('Failed to find email input field')

                logger.info(f'Profile {profile_num} | Confirming email')
                m = MailGW()
                await mail_field.type(m.email)

                await asyncio.sleep(0.3)

                submit_button = await page.waitForSelector('.acef_btn')
                await submit_button.click()

                msg = m.wait_for_message()
                pattern = r'https://bongacams.com/members/confirm-email/[a-zA-Z0-9]+'
                match = re.search(pattern, msg)
                link = match.group().replace('bongacams.com', 'ru3.bongacams.com')
                await page.goto(link)

                logger.info(f'Profile {profile_num} | Email confirmed')

                element = await page.waitForSelector('.js-tokens')

                await asyncio.sleep(1.5)
                tokens = await page.evaluate('(element) => element.textContent.trim()', element)
                tokens = int(tokens)

                if tokens:
                    logger.info(f'Profile {profile_num} | Tokens received.')

                    with file_lock:
                        with open('bongas.txt', 'a') as f:
                            f.write(f'{username}:{password}\n')
                else:
                    logger.error(f'Profile {profile_num} | No tokens received')

                await browser.close()
                clz = True

            except Exception as e:
                logger.error(f'Profile {profile_num} | Error: {str(e)}')
                raise

            finally:
                try:
                    if not clz:
                        await browser.close()
                except:
                    pass
                logger.info(f'Profile {profile_num} | Closing profile')
                dolphin.close_profile(profile_id)
                api.delete_profiles([profile_id])

        try:
            asyncio.run(main(i+1, username, password))
        except Exception as e:
            logger.error(f'Profile {i+1} | Error: {str(e)}. Deleting profile and moving to next one.')
            dolphin.close_profile(profile_id)
            api.delete_profiles([profile_id])

    except Exception as e:
        logger.error(f'Profile {i+1} | Error: {str(e)}. Deleting profile and moving to next one.')
        api.delete_profiles([profile_id])

with ThreadPoolExecutor(max_workers=5) as executor:
    futures = []
    for i, proxy in enumerate(valid_proxies):
        future = executor.submit(process_profile, i, proxy)
        futures.append(future)

        if len(futures) == 5:            
            for future in futures:
                future.result()
            futures = []

BersXx commented 5 months ago

Thank you for sharing this DedInc, learning a lot from you mate, for some reason no matter what I do with my codes I must interact with dolphin anty software every like 1 hour for the API to stay connected, I am still looking further into this

I am reading your code at the moment <3

BersXx commented 5 months ago

DedInc what is this, and do you think I should allow it? it's from dolphin anty, but windows firewall is blocking, this might be my problem not sure:

BersXx commented 5 months ago

after looking into it more I think the issue is coming from here DedInc, this is from selenium_dolphin.py:

Sorry of I am confusing you a lot here

def get_driver(options=Options(), port=9222):
    system = platform.system()
    architecture = platform.machine()
    driver_path = select_driver_executable(system, architecture)

    if not os.path.exists(driver_path):
        driver_content = get_dolphin_driver()
        unzip_driver_from_memory(driver_content)
        driver_path = select_driver_executable(system, architecture)

    options.add_experimental_option('debuggerAddress', f'127.0.0.1:{port}')
    driver = webdriver.Chrome(service=Service(driver_path), options=options)
    return driver

DedInc commented 5 months ago

try using pyppeteer instead of selenium.

BersXx commented 5 months ago

I will give it a try today DedInc, I have a free plan with 30 profiles, do you think this will affect the API limits maybe? cuz i am getting a lot of 401 errors, if I got the base 100 profiles you think this might solve the issue?

DedInc / pyanty

dolphin-selenium getting inactive after a while #18