search

oseymour / ScraperFC

Python package for scraping soccer data from a variety of sources
GNU General Public License v3.0
222 stars 49 forks source link

Capology scrape_salaries not working #39

Closed rubeenvm2 closed 5 days ago

rubeenvm2 commented 5 months ago

I'm trying to scrape salaries as follows: scraper = sfc.Capology() out = scraper.scrape_salaries(2019, "La Liga", "eur") scraper.close() But I'm getting an strange error that could be a bug or maybe that 10 seconds are not enough to find the 'All' button. You could add a variable to configure the max seconds in order to avoid this if that's causing the error:

TimeoutException Traceback (most recent call last) Cell In[196], line 9 7 for year in years: 8 a = 1 ----> 9 out = scraper.scrape_salaries(2019, "La Liga", "eur") 11 # Close the scraper 12 scraper.close()

File ~\AppData\Roaming\Python\Python38\site-packages\ScraperFC\Capology.py:81, in Capology.scrape_salaries(self, year, league, currency) 79 while not done: 80 try: ---> 81 all_btn = WebDriverWait( 82 self.driver, 83 10, 84 ).until(EC.element_to_be_clickable( 85 (By.LINK_TEXT, 'All') 86 )) 87 all_btn.click() 88 done = True

File ~\AppData\Roaming\Python\Python38\site-packages\selenium\webdriver\support\wait.py:105, in WebDriverWait.until(self, method, message) 103 if time.monotonic() > end_time: 104 break --> 105 raise TimeoutException(message, screen, stacktrace)

TimeoutException: Message: Stacktrace: GetHandleVerifier [0x00007FF671CF2142+3514994] (No symbol) [0x00007FF671910CE2] (No symbol) [0x00007FF6717B76AA] (No symbol) [0x00007FF671801860] (No symbol) [0x00007FF67180197C] (No symbol) [0x00007FF671844EE7] (No symbol) [0x00007FF67182602F] (No symbol) [0x00007FF6718428F6] (No symbol) [0x00007FF671825D93] (No symbol) [0x00007FF6717F4BDC] (No symbol) [0x00007FF6717F5C64] GetHandleVerifier [0x00007FF671D1E16B+3695259] GetHandleVerifier [0x00007FF671D76737+4057191] GetHandleVerifier [0x00007FF671D6E4E3+4023827] GetHandleVerifier [0x00007FF671A404F9+689705] (No symbol) [0x00007FF67191C048] (No symbol) [0x00007FF671918044] (No symbol) [0x00007FF6719181C9] (No symbol) [0x00007FF6719088C4] BaseThreadInitThunk [0x00007FFA5659257D+29] RtlUserThreadStart [0x00007FFA57C2AA58+40]

oseymour commented 3 months ago

Hi @rubeenvm2. Same as the other issue you opened; I'll get to fixing this when I have some time for ScraperFC again!

oseymour commented 5 days ago

@rubeenvm2 This should be working in v3.0.0!