Transfermarkt Module Not Finding Tags

[acdaniel864]

Module: Transfermarkt

Description of problem: Unable to retrieve information with transfermarkt module.

Code:

import ScraperFC as sfc
import traceback

scraper = sfc.Transfermarkt()
try:
    # Scrape the table
    player_links = scraper.get_players(year=2022, league="EPL")
except:
    traceback.print_exc()
finally:
    scraper.close()

Error message:

AttributeError Traceback (most recent call last) Cell In[7], line 4 1 import ScraperFC as sfc 2 import traceback ----> 4 scraper = sfc.Transfermarkt() 5 try: 6 # Scrape the table 7 player_links = scraper.get_player_links(year=2023, league="La Liga")

File ~/Desktop/github/scraperfc/ScraperFC/Transfermarkt.py:38, in Transfermarkt.init(self) 34 # Press the ACCEPT ALL button 35 soup = BeautifulSoup(self.driver.page_source, 'html.parser') 36 accept_all_button = self.driver.find_element( 37 By.XPATH, ---> 38 xpath_soup(soup.find('button', {'aria-label': 'ACCEPT ALL'})) 39 ) 40 self.driver.execute_script('arguments[0].click()', accept_all_button) 41 # Switch back to the main window

File ~/Desktop/github/scraperfc/ScraperFC/shared_functions.py:696, in xpath_soup(element) 667 """ Generate xpath from BeautifulSoup4 element. 668 669 I shamelessly stole this from https://gist.github.com/ergoithz/6cf043e3fdedd1b94fcf. (...) 693 "/doc/elm[2]" 694 """ 695 components = [] --> 696 child = element if element.name else element.parent 697 for parent in child.parents: # type: bs4.element.Tag 698 siblings = parent.find_all(child.name, recursive=False)

AttributeError: 'NoneType' object has no attribute 'name'

[oseymour]

They must have changed their cookie accepting popup. I'm working on other projects right now but this is on the to-do list for when I have time for ScraperFC again!

[joaomcalves]

Hi, were you able to solve this issue?

[oseymour]

@JoaoMCAlves haven't had a chance to look at this yet, sorry. Spent the last month prepping for ACL and meniscus surgery which happened yesterday. I'm planning on using my time laid up in bed to fix this.

[joaomcalves]

Hello @oseymour , I'm sorry to hear that, I hope you have a speedy recovery. Congratulations on the excellent work you've been doing and thank you for the effort.

[joaomcalves]

Hello @oseymour , hope you are reconvering well! Any news on this?

[oseymour]

Yep, still working on all the other changes for the next release before it's pushed to PyPI. But if you really need the changes now they're on the unit tests branch.

[joaomcalves]

Thanks @oseymour!

[oseymour]

Should be fixed in v3.0.0 now. Please re-open if it's not!

[joaomcalves]

Thanks @oseymour ! I'm going to try this afternoon.

[joaomcalves]

It worked @oseymour , thank you very much!

[oseymour]

Nice! Sorry it took me so long to fix and release!

[joaomcalves]

No problem! I know that it always takes time. I just bought you a coffee 🙂