search

andrewlamm / csgodoku

csgodoku: an immaculate grid for cs:go
https://csgodoku.onrender.com/
3 stars 0 forks source link

Pull team historical data from Liquipedia's API instead of scraping HLTV #3

Open 18swenskiq opened 1 year ago

18swenskiq commented 1 year ago

I'm an avid pro CS fan, and I really like the project. Honestly the only thing I dislike about this currently is that there are quite a lot of data discrepancies in the team data, meaning some answers that would be correct just aren't there (for example the other day the one with 3DMax was missing players, notably Allu).

HLTV just does not have full historical team data, meanwhile Liquipedia basically has all of it you could ever want, and it is as wholly accurate as a public source can be.

So my suggestion here is to pull historical team data from Liquipedia's API instead to improve accuracy

andrewlamm commented 1 year ago

Hey! I'm glad you enjoyed this project! I agree that using Liquipedia's API will give a more comprehensive team list but I'm afraid that will make the script that gets the data take extremely long to run and I'm not sure it is necessary. This is because:

  1. I have updated the script to use the teams from the player's entire match history. So, now the script will not miss any teams that it they were listed as stand-ins for (this fixed the issue with allu not playing in 3DMax).
  2. This (point 1) does not account for matches that don't show up on HLTV, so teams that the player played on in these matches will not be listed. However, the puzzles will never use these teams, so I believe this is okay.

As such, I do not believe that the increase in script runtime is worth it. However, if you disagree (especially if you find more errors where a team is missing some players) please let me know!