SeleniumHQ / selenium

A browser automation framework and ecosystem.
https://selenium.dev
Apache License 2.0
30.63k stars 8.18k forks source link

Aw, Snap! Something went wrong while displaying this webpage. Error code: Out of Memory[šŸ› Bug]: #13587

Closed dheeraj009joshi closed 8 months ago

dheeraj009joshi commented 8 months ago

What happened?

while running my python test script it basically work's fine in the beginning but eventually after some time it started slowing down the page loading and after an hour or somthing it show me this in my webdriver WhatsApp Image 2024-02-15 at 08 37 49_75150c54

How can we reproduce the issue?

can you plese resolve this issue so that i can run my script easily  i don't want t his memory issue to come 

here is my script :- 
import datetime
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import TimeoutException
import undetected_chromedriver as uc
import time 
options = webdriver.ChromeOptions()

driver = webdriver.Chrome()

def calculate_days_difference(given_date_str):
    # Step 1: Obtain the current date
    current_date =datetime.date.today()

    # Step 2: Determine the given date
    given_date = datetime.datetime.strptime(given_date_str, "%m/%d/%y").date()

    # Step 3: Calculate the difference in days
    days_difference = (current_date - given_date).days

    return days_difference
driver.get("https://group1auto.onelogin.com/")
# trying to login 
time.sleep(5)
driver.find_element(By.XPATH,'//*[@id="username"]').send_keys(username)
driver.find_element(By.XPATH,'//*[@id="root"]/div/div/div[2]/div[1]/div[2]/form/div/div[3]/div/button').click()
time.sleep(5)
driver.find_element(By.XPATH,'//*[@id="password"]').send_keys(password)
driver.find_element(By.XPATH,'//*[@id="root"]/div/div/div[2]/div[1]/div[2]/form/div/div[3]/div/button').click()
verify=0
while verify==0:
    try:
        driver.find_element(By.XPATH,'//*[@id="root"]/div/div/div[2]/div[1]/div[2]/div/div[2]/div[2]/button').click()
        verify=1
    except:
        pass
time.sleep(10)
driver.find_element(By.XPATH,'//*[@id="root"]/div/main/div/div/p[2]/button').click()
time.sleep(10)
# # # # login done 
all_oprions_after_login=driver.find_elements(By.XPATH,'//*[@id="apps-view-container"]/div[2]/div/div/div/div/a/div/div[1]/div')
for opt in all_oprions_after_login:
    if opt.text=='ELEAD SSO SAML':
        opt.click()
time.sleep(10)   
driver.switch_to.window(driver.window_handles[1])

try:
    # driver.switch_to.window(driver.window_handles[2])
    # driver.close()
    driver.switch_to.window(driver.window_handles[2])
except:
    pass

details_leads=open(f"All_leads_{datetime.date.today()}_details_all_in_one.csv","a")
details_leads.write("Name,cell,email,Created_date,Created_time,vehicle,PrimarySalesPerson,LastDateOf_contact,LastDateOf_contact_time,Number_of_times_SalesPerson_contacted,Number_of_times_customer_contacted,Pending_Replies,Pending_Replies_Text,avg_daily_contacted,sales_status,up_type,source,sub_source,Lead_url\n")
f=open(f'all_leade_all_in_one_{datetime.date.today()}.csv','r')
import json
all_urls=[]
for i in f.readlines():
    i=i.replace("'",'"')
    try:
        json_object = json.loads(i)
        print(json_object['Url'])
        all_urls.append(json_object['Url'])
    except:
        pass
print(len(all_urls))

# Set the batch size
batch_size = 500  # You can adjust this value based on your needs

# all_urls=['https://group1.eleadcrm.com/evo2/fresh/eLead-V45/elead_track/NewProspects/OpptyDetails.aspx?lPID=22831066&lDID=21993180','https://group1.eleadcrm.com/evo2/fresh/eLead-V45/elead_track/NewProspects/OpptyDetails.aspx?lPID=22726119&lDID=21869119']
# all_urls=['https://group1.eleadcrm.com/evo2/fresh/elead-v45/elead_track/NewProspects/OpptyDetails.aspx?lPID=22895748&lDID=22055520','https://group1.eleadcrm.com/evo2/fresh/eLead-V45/elead_track/NewProspects/OpptyDetails.aspx?lPID=10901982&lDID=22075070','https://group1.eleadcrm.com/evo2/fresh/elead-v45/elead_track/NewProspects/OpptyDetails.aspx?lPID=22841517&lDID=22018748','https://group1.eleadcrm.com/evo2/fresh/elead-v45/elead_track/NewProspects/OpptyDetails.aspx?lPID=22775406&lDID=21927585']
# all_urls=['https://group1.eleadcrm.com/evo2/fresh/elead-v45/elead_track/NewProspects/OpptyDetails.aspx?lPID=22998940&lDID=22169272']
count_leads_processed=0
count_leads_written_in_csv=0

batch_no=1
url_batches = [all_urls[i:i + batch_size] for i in range(0, len(all_urls), batch_size)]
try:
    # Process each batch
    for batch in url_batches:
        print("batch no = "+str(batch_no))
        try:
            for url in batch:
                try:
                    Lead_url=url
                    driver.get(url)
                    count_leads_processed+=1
                    print(f"Leads processes :- {count_leads_processed}")
                    # time.sleep(5)
                    vehicle=''
                    PrimarySalesPerson=''
                    try:
                        view_previous_data=driver.find_element(By.XPATH,'//*[@id="OpportunityPanel_ViewPrevOpptyLink"]/span').click()
                        # print("got it ")
                        time.sleep(1)
                        all_oprions_for_sales_statue=driver.find_elements(By.XPATH,'//*[@id="OpportunityPanel_StatusDropdown"]/option')
                        for option in all_oprions_for_sales_statue:

                            if option.is_selected():
                                sales_status=option.text
                                # print(sales_status)

                    except:

                        all_oprions_for_sales_statue=driver.find_elements(By.XPATH,'//*[@id="OpportunityPanel_StatusDropdown"]/option')
                        for option in all_oprions_for_sales_statue:

                            if option.is_selected():
                                sales_status=option.text
                                # print(sales_status)

                    opportunity_section_text=driver.find_element(By.XPATH,'//*[@id="OpportunityPanel_ActiveOpptyPanel"]').text
                    try:
                        up_type=driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[2]/table/tbody/tr[6]/td[2]').text
                    except:
                        up_type=driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[3]/table/tbody/tr[6]/td[2]').text

                    # print("Up_type=  "+up_type)             # /html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[3]/table/tbody/tr[6]/td[2]
                    try:
                        if "Source" in driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[2]/table/tbody/tr[7]').text:
                            source=driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[2]/table/tbody/tr[7]/td[2]').text
                        elif "Source" in driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[2]/table/tbody/tr[8]').text:
                            source=driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[2]/table/tbody/tr[8]/td[2]').text
                        else:
                            source=""
                    except:
                        if "Source" in driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[3]/table/tbody/tr[7]').text:
                            source=driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[3]/table/tbody/tr[7]/td[2]').text
                        elif "Source" in driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[3]/table/tbody/tr[8]').text:
                            source=driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[3]/table/tbody/tr[8]/td[2]').text
                        else:
                            source=""
                    # print("source=  "+source)
                    try:                   
                        if "Sub-source" in driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[2]/table/tbody/tr[8]').text:
                            sub_source=driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[2]/table/tbody/tr[8]/td[2]').text
                        elif "Sub-source" in driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[2]/table/tbody/tr[8]').text:
                            sub_source=driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[2]/table/tbody/tr[8]/td[2]').text
                        else:
                            sub_source=""
                    except Exception as e :
                        print(e)
                        pass
                        # if "Sub-source" in driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[3]/table/tbody/tr[9]').text:
                        #     sub_source=driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[3]/table/tbody/tr[9]/td[2]').text
                        # elif "Sub-source" in driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[3]/table/tbody/tr[9]').text:
                        #     sub_source=driver.find_element(By.XPATH,'/html/body/form/div[3]/table[2]/tbody/tr[2]/td[4]/div[1]/div[1]/div[3]/table/tbody/tr[9]/td[2]').text
                        # else:
                        #     sub_source=""
                    # print("Sub-source=  "+sub_source)

        # # # # # # Getting General Details   
                    # print("getting details")
                    try:
                        vehicle=driver.find_element(By.XPATH,'//*[@id="OpportunityPanel_ActiveOpptyPanel"]/table/tbody/tr[2]/td[3]').text.replace("Add Opportunity",'')
                    except:
                        pass
                    try:
                        PrimarySalesPerson=driver.find_element(By.XPATH,'//*[@id="OpportunityPanel_ActiveOpptyPanel"]/table/tbody/tr[5]/td[2]/table/tbody/tr[1]/td[1]').text
                    except:
                        pass 
                    name=driver.find_element(By.XPATH,'//*[@id="CustomerAjaxPanel"]/table/tbody/tr[2]/td[3]').text
                    try:
                        cell=driver.find_element(By.XPATH,'//*[@id="CustomerAjaxPanel"]/table/tbody/tr[6]/td[2]/div[1]/table/tbody/tr/td[1]').text
                    except:
                        cell=''
                    email=driver.find_element(By.XPATH,'//*[@id="CustomerAjaxPanel"]/table/tbody/tr[8]/td[2]').text 
                    driver.switch_to.frame("tabsTargetFrame")
                    try:
                        Created=driver.find_element(By.XPATH,'//*[@id="gvOpptyHistory"]/tbody/tr[1]/td[2]').text
                    except:
                        Created=''

        # # # # # # contacted details   
                    Number_of_times_SalesPerson_contacted=0
                    Number_of_times_customer_contacted=0
                    Pending_Reply=False
                    Pending_Reply_text=''
                    all_contacts=driver.find_elements(By.XPATH,'/html/body/form/table/tbody/tr[4]/td/div/div/table/tbody/tr[2]/td/div/table/tbody/tr')
                    # print(len(all_contacts))  #################/html/body/form/table/tbody/tr/td/div/div/table/tbody/tr/td/div/table/tbody/tr/td[5]/div
                    date_of_contacts=[]

                    for i in all_contacts:

                        # print(" i am in the loop ")
                        # Times lead responded
                        try:

                            contect_type=i.text.split("\n")[1]
                            contect_outcome=i.text.split("\n")[2]
                            # print("typr_contacted ="+contect_type)
                            if 'Read Email' in contect_type or (contect_type=='Text Message' and contect_outcome == "Received"):
                                Number_of_times_customer_contacted+=1
                        except:
                            pass

                        # Times SalesPErson contacted
                        try:
                            # print(i.find_element(By.XPATH,"td[6]").text)
                        # print("sales person contact ="+ i.text.split('\n')[3])
                            aa=i.find_element(By.XPATH,"td[6]").text.split(",")[0]
                            # print(aa)
                            if aa==PrimarySalesPerson.split(",")[0]:
                                date_of_contacts.append((i.text).split("\n")[0])
                                Number_of_times_SalesPerson_contacted+=1
                        except:
                            pass

                    try:
                        # print(" i am inside the true false section ")
                        contect_type=all_contacts[1].text.split("\n")[1]
                        contect_outcome=all_contacts[1].text.split("\n")[2]
                        # print("typr_contacted ="+contect_type)
                        if 'Read Email' in contect_type or (contect_type=='Text Message' and contect_outcome == "Received"):
                                Pending_Reply=True
                                if 'Email' in  contect_type:
                                    pass
                                    # all_contacts[1].find_element(By.XPATH,"td[7]/div/a").click()
                                    # time.sleep(2)
                                    # main_window = driver.current_window_handle
                                    # # print(main_window)
                                    # popup_window = None
                                    # for handle in driver.window_handles:
                                    #     if handle != main_window:
                                    #         popup_window = handle
                                    # pop=driver.switch_to.window(popup_window)
                                    # # print("pop up window + "+popup_window)
                                    # text=driver.find_element(By.XPATH,'/html/body/form/div[4]/div[1]').text
                                    # Pending_Reply_text=text.replace(",",' ').replace('\n',' ')
                                    # driver.close()
                                    # # print(text)
                                    # driver.switch_to.window(main_window)
                                    # print("main window  +"+main_window)
                                    # print("switching to main window")
                                else:       
                                    Pending_Reply_text=all_contacts[1].find_element(By.XPATH,"td[5]/div").text
                    except:
                        pass

        # # # # # # Last Date of contacted
                    try:
                        LastDateOf_contact=date_of_contacts[0]
                    except:
                        LastDateOf_contact=''

        # # # # # #  Average contacted to a person( Lead)
                    try:
                        total_days_after_creation=calculate_days_difference(Created.split(" ")[0])
                        avg_daily_contacted=len(all_contacts)/total_days_after_creation
                    except:
                        avg_daily_contacted=0

        # # # # # # validating the data and adding it to file
                    if vehicle !='' or PrimarySalesPerson !='':
                        # print("OPPORTUNITY available")
                        # print(name)
                        # print(cell)
                        # print(email)
                        # print(vehicle)
                        # print(PrimarySalesPerson)
                        # print(LastDateOf_contact)
                        # print(Number_of_times_SalesPerson_contacted)
                        # print(Number_of_times_customer_contacted)
                        # print(Pending_Reply)
                        # print(Pending_Reply_text)
                        vehicle=vehicle.replace("\n"," ")
                        PrimarySalesPerson=PrimarySalesPerson.replace(","," ")
                        formatted_avg_daily_contacted = "{:.2f}".format(avg_daily_contacted)
                        # print(formatted_avg_daily_contacted)
                        # print( "testing the data ")
                        # print(Created.split(" ")[0])
                        # print(Created.split(" ")[1]+" "+Created.split(" ")[2])
                        try:
                            Lase_date_contact_date=LastDateOf_contact.replace("keyboard_arrow_down ",'').split(" ")[0]
                            Lase_date_contact_date_time=LastDateOf_contact.replace("keyboard_arrow_down ",'').split(" ")[1]+" "+LastDateOf_contact.replace("keyboard_arrow_down ",'').split(" ")[2]
                        except:
                            Lase_date_contact_date=''
                            Lase_date_contact_date_time=''

                        details_leads.write(f'{name},{cell},{email},{Created.split(" ")[0]},{Created.split(" ")[1]+" "+Created.split(" ")[2]},{vehicle},{PrimarySalesPerson},{Lase_date_contact_date},{Lase_date_contact_date_time},{Number_of_times_SalesPerson_contacted},{Number_of_times_customer_contacted},{Pending_Reply},{Pending_Reply_text},{formatted_avg_daily_contacted},{sales_status},{up_type},{source},{sub_source},{Lead_url}'+'\n')
                        count_leads_written_in_csv+=1
                        print(f"Leads written in csv :- {count_leads_written_in_csv}")
                    # time.sleep(5)    
                except Exception as e :
                    print(e)
                    pass
            # Execute JavaScript to clear local storage
            driver.execute_script("window.localStorage.clear();")
            driver.execute_script("window.sessionStorage.clear();")
            print("local storage cleared")

            driver.execute_script("window.open();")
            driver.switch_to.window(driver.window_handles[-1])

            print("new tab opened")
            time.sleep(5)
            # Switch to the new tab
            driver.switch_to.window(driver.window_handles[-2])
            driver.close()
            # Switch back to the new tab
            driver.switch_to.window(driver.window_handles[-1])
            time.sleep(3)
            driver.get(url)
            batch_no+=1

        except TimeoutException:
                print("Timed out waiting for element to be present. Switching to new tab.")
                time.sleep(10)
                driver.execute_script("window.localStorage.clear();")
                driver.execute_script("window.sessionStorage.clear();")
                print("local storage cleared")

                driver.execute_script("window.open();")
                driver.switch_to.window(driver.window_handles[-1])

                print("new tab opened")
                time.sleep(5)
                # Switch to the new tab
                driver.switch_to.window(driver.window_handles[-2])
                driver.close()
                # Switch back to the new tab
                driver.switch_to.window(driver.window_handles[-1])
                time.sleep(3)
                driver.get(url)
        except Exception as err:
                print(err)
                pass

except Exception as err:
    print(err)   
    pass

Relevant log output

no logs available

Operating System

windows 11 pro

Selenium version

python 3.12

What are the browser(s) and version(s) where you see this issue?

chrome 121

What are the browser driver(s) and version(s) where you see this issue?

chromeDriver 121

Are you using Selenium Grid?

No response

github-actions[bot] commented 8 months ago

@dheeraj009joshi, thank you for creating this issue. We will troubleshoot it as soon as we can.


Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

diemol commented 8 months ago

There are reports where Chrome 121 is using too much memory. Try Chrome 120 or wait a few days for Chrome 122.

github-actions[bot] commented 8 months ago

šŸ’¬ Please ask questions at:

github-actions[bot] commented 7 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.