search

HWYA / shapeOfTeam

3 stars 2 forks source link

Aaron Seydel POB Scrape Breaks #93

Open sunsheeppoplar opened 7 years ago

sunsheeppoplar commented 7 years ago

http://www.transfermarkt.com/aaron-seydel/profil/spieler/187499

NoMethodError: undefined method `children' for nil:NilClass /home/sunsheeppoplar/WebDev/fun-projects/shapeOfTeam/lib/tasks/scrape_player_place_of_birth.rake:24:in `block (3 levels) in <top (required)>

Need to refactor immediately. Offending line is line 22 of scrape_player_place_of_birth:

parent_span = doc.at_css('.hide-for-small>span.dataValue')

Maybe don't look in .hide-for-small since they usually have a corresponding value for standard window sizes

sunsheeppoplar commented 7 years ago

Not dissimilar to what #53 tried to address with missing POBs. Players who are young seem to be thrown into an HTML structure that's not kind to our old scrape task.

sunsheeppoplar commented 7 years ago

Seems that this might be caused because he's a call-up from the second league.

http://www.transfermarkt.com/1-fsv-mainz-05/startseite/verein/39/saison_id/2016

sunsheeppoplar commented 7 years ago

It seems to be that the issue is, indeed, from that above-mentioned parent_span. I originally had it as doc.css but later changed it to doc.at_css which couldn't locate something that had children. The former does find something and thus obfuscated this error from us. It would look through the children and none of the if statements would be hit, so the player would just save.

These do seem to be players who have different HTML formatting on their pages because they're young and recent call ups into the first team. We remedied it originally with the scrape_player_missing_pob.rake Now we just have to decide how to address this. This is basically the effect we want, but we were just lucky that the css selector didn't break earlier