PennyDreadfulMTG / Penny-Dreadful-Tools

A suite of tools for the Penny Dreadful MTGO community
https://pennydreadfulmagic.com
MIT License
41 stars 28 forks source link

Gatherling scraper failure #4108

Closed bakert closed 6 years ago

bakert commented 6 years ago
Fetching http://gatherling.com/eventreport.php?event=Penny Dreadful Saturdays 6.11 (cache ok)
POSTing to http://gatherling.com/deckdl.php with {'id': '54808'}
Traceback (most recent call last):
  File "run.py", line 77, in <module>
    run()
  File "run.py", line 32, in run
    task(sys.argv)
  File "run.py", line 70, in task
    s.scrape()
  File "/home/discord/decksite/decksite/scrapers/gatherling.py", line 24, in scrape
    i = tournament(url, name)
  File "/home/discord/decksite/decksite/scrapers/gatherling.py", line 50, in tournament
    return add_decks(dt, competition_id, ranks, s)
  File "/home/discord/decksite/decksite/scrapers/gatherling.py", line 58, in add_decks
    d = tournament_deck(cells, competition_id, dt, ranks)
  File "/home/discord/decksite/decksite/scrapers/gatherling.py", line 120, in tournament_deck
    d['cards'] = decklist.parse(fetcher.internal.post(gatherling_url('deckdl.php'), {'id': gatherling_id}))
  File "/home/discord/decksite/decksite/scrapers/decklist.py", line 41, in parse
    n, name = parse_line(lines.pop(0))
  File "/home/discord/decksite/decksite/scrapers/decklist.py", line 14, in parse_line
    raise InvalidDataException('No number specified with `{line}`'.format(line=line))
shared.pd_exception.InvalidDataException: No number specified with `Sideboard`
silasary commented 6 years ago

The deck in question has zero cards.

This is another point towards using the JSON API to download decklists. http://gatherling.com/ajax.php?deck=54808

bakert commented 6 years ago

I fixed this for now. JSON is nice though.