search

agoragames / bnet_scraper

A Nokogiri-based scraper of Battle.net profiles. Currently this only includes Starcraft2.
MIT License
22 stars 7 forks source link

Date format are not handled. #17

Closed mkmkmkmk closed 11 years ago

mkmkmkmk commented 11 years ago

Hello there, seems to be a nice tool you are doing. I am trying to use it.

It seems that date format are not handled well It have errors (see details below) for achievments dates:

When 'us' region page format is : mm/dd/yyyy (Works well here: http://us.battle.net/sc2/en/profile/2377239/1/Demon/achievements/)

In several others region/locale : dd/mm/yyyy (Fails here, even if local put to en: http://eu.battle.net/sc2/en/profile/1229243/1/Stephano/achievements/)

Its more like a bnet website issue... (since 'en' locale is set but not handled) but still as such one cannot scrap achievements properly.

Thanks a lot! Cheers

C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scraper/starcraf t2/achievement.rb:20:in new': invalid date (ArgumentError) from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/achievement.rb:20:inconvert_date' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/achievement.rb:15:in earned=' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/achievement.rb:10:inblock in initialize' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/achievement.rb:9:in each_key' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/achievement.rb:9:ininitialize' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/achievement_scraper.rb:78:in new' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/achievement_scraper.rb:78:inextract_recent_achievement' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/achievement_scraper.rb:65:in block in scrape_recent' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/achievement_scraper.rb:64:intimes' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/achievement_scraper.rb:64:in scrape_recent' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/achievement_scraper.rb:41:inscrape' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/profile.rb:18:in achievements' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/bnet_scraper-0.6.0/lib/bnet_scr aper/starcraft2/profile.rb:22:inrecent_achievements' from C:/Users/xxx/Desktop/a.rb:17:in `

'

cadwallion commented 11 years ago

I will take a look at this tonight. It's possible that the VCR cassettes are outdated and Blizzard has changed the markup on me.

cadwallion commented 11 years ago

Writing a fix for this at the moment, it looks like it's not locale based but instead region based (eu.battle.net, us.battle.net, etc). I am adding in date parsing logic by domain.