probberechts / soccerdata

⛏⚽ Scrape soccer data from Club Elo, ESPN, FBref, FiveThirtyEight, Football-Data.co.uk, FotMob, Sofascore, SoFIFA, Understat and WhoScored.
https://soccerdata.readthedocs.io/en/latest/
Other
526 stars 89 forks source link

Add support for scraping Sofascore #505

Closed ksbharaj closed 2 months ago

ksbharaj commented 3 months ago

Hi All. I have included some code to scrape Sofascore. It's fairly basic now, focussed only on the top 5 leagues. I can add more functions going forward. Happy to get any feedback to help me improve it.

probberechts commented 3 months ago

Hi. Thanks for your contribution. Does Sofascore provide any data that is not yet covered by the scrapers that are already implemented? I am somewhat hesitant to add more sources that provide exactly the same data, because each data source comes with a maintenance cost.

ksbharaj commented 3 months ago

Apologies for my delayed response. Github's stopped emailing me notifications - my responses will be quicker next time. Yes, Sofascore does. I've listed its distinguishing qualities below:

I understand your concerns on the maintenance side. However, I'm happy to manage it for SofaScore going forward if possible.

ksbharaj commented 3 months ago

Also worth mentioning is that I haven't included the listed features above in this initial commit- I intended for the initial commit to be simple. I can add it in the coming days- should not take too much time/effort.

probberechts commented 3 months ago

Thanks for the clarification. For me, the most important requirement is that a few tests are added and that the docs are updated. Extra features can be added later.

ksbharaj commented 3 months ago

I agree. I will work towards that then. Thank you!

codecov-commenter commented 3 months ago

Codecov Report

Attention: Patch coverage is 93.61702% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 65.15%. Comparing base (69fb0de) to head (a22d18a). Report is 51 commits behind head on master.

Files Patch % Lines
soccerdata/sofascore.py 95.50% 2 Missing and 2 partials :warning:
soccerdata/_common.py 50.00% 1 Missing and 1 partial :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #505 +/- ## ========================================== + Coverage 59.33% 65.15% +5.81% ========================================== Files 11 13 +2 Lines 1633 2003 +370 Branches 336 399 +63 ========================================== + Hits 969 1305 +336 - Misses 610 631 +21 - Partials 54 67 +13 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

ksbharaj commented 3 months ago

Hi. Please let me know if there's anything else required to merge this PR. I've edited the docs too. Awaiting feedback and/or approval, and I can add more features. Also, let me know if anything else needs to be added or done differently. Thanks!

ksbharaj commented 3 months ago

Thank you for reviewing! I'll look to get these addressed asap (next couple of days at most)

probberechts commented 2 months ago

Could you fix the merge conflicts, please? The tests currently fail due to an issue with pipx installs in GHA but I've just fixed this in the master branch.