search

andrexsaddler / vlr-api

unofficial api of VLR.gg
0 stars 0 forks source link

issue with scrape #1

Open andrexsaddler opened 3 years ago

andrexsaddler commented 3 years ago

@josephsookim if you have a chance can you checkout the code? issue with listing all the items from vlr.gg page and its rankings page

josephsookim commented 3 years ago

For the rankings, you will need to individually scrape a page for each different region since it won't be together. It seems like you are only scraping the na page atm.

World = https://www.vlr.gg/rankings Europe = https://www.vlr.gg/rankings/europe NA = https://www.vlr.gg/rankings/north-america Asia-Pacific = https://www.vlr.gg/rankings/asia-pacific Latin America = https://www.vlr.gg/rankings/latin-america Oceania = https://www.vlr.gg/rankings/oceania Korea = https://www.vlr.gg/rankings/korea MENA = https://www.vlr.gg/rankings/mena

josephsookim commented 3 years ago

Also, for featured, are you trying to get the top article on the site? For example, TenZ and Sentinels — A match made in heaven?

andrexsaddler commented 3 years ago

Also, for featured, are you trying to get the top article on the site? For example, TenZ and Sentinels — A match made in heaven?

actually i was trying to scrape https://www.vlr.gg/news, featured route was the easiest for me to scrape

however the news route is the main thing i am working on

josephsookim commented 3 years ago

When I was testing your code earlier, I ran into some issues. You might need to double check the featured one.

Error was NoneType object has no attribute text, specifically caused by: title = video.find('div', class_="wf-spoiler-visible").text.strip()

What seems to be the issue with the news route? Article Title and URL scraping seems to be working fine.

andrexsaddler commented 3 years ago

When I was testing your code earlier, I ran into some issues. You might need to double check the featured one.

Error was NoneType object has no attribute text, specifically caused by: title = video.find('div', class_="wf-spoiler-visible").text.strip()

What seems to be the issue with the news route? Article Title and URL scraping seems to be working fine.

if you look closely, it only grabs first article in wf-card but not the subsequent older articles