search

j-andrews7 / kenpompy

A simple yet comprehensive web scraper for kenpom.com.
https://kenpompy.readthedocs.io/en/latest/?badge=latest
GNU General Public License v3.0
70 stars 21 forks source link

"Conference only" filter in KenPom Scouting Reports #63

Closed ddogwoofwoof closed 10 months ago

ddogwoofwoof commented 10 months ago

In KenPom's scouting reports, there is a "Conference only" filter. Unfortunately, I don't see this filter option using the summary.py file. Do you know if this is possible to include? I inspected KenPom's website, but struggled to find the elements that include this info. Thank you! Derek

j-andrews7 commented 10 months ago

Those are tricky, as it requires interacting with the page in a difficult way. We'd like to add them, but it's not straight forward. I can't remember if we figured out a way to do so on the other pages yet or not.

We'll probably push a release before the season kicks off fully to adjust for changes he's made to the site since the end of last season.

On Mon, Oct 23, 2023, 3:23 PM ddogwoofwoof @.***> wrote:

In KenPom's scouting reports, there is a "Conference only" filter. Unfortunately, I don't see this filter option using the summary.py file. Do you know if this is possible to include? I inspected KenPom's website, but struggled to find the elements that include this info. Thank you! Derek

— Reply to this email directly, view it on GitHub https://github.com/j-andrews7/kenpompy/issues/63, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOAQNAZ746EWFAYCQH2GXTYA3G2TAVCNFSM6AAAAAA6MUMIZGVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE2TOOJWGI3TONI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

ddogwoofwoof commented 10 months ago

That makes sense. I'll take a stab at it as well and create a pull request if I can make progress. Thank you!

esqew commented 10 months ago

I believe what you're referring to is something that has already had work put forward in #40. It's currently stalled for a variety of reasons, the biggest of which being that I'm not convinced we should take additional overhead to execute arbitrary JavaScript for support of such a small sliver of the site.

I have a local branch where I've tried to do this purely with the current dependency set and parsing some of the JS content with a thoughtful RegExp but it's not yet all the way there.

If you are so inclined, please feel free to try to pull together a PR and we'd be more than happy to consider it.

esqew commented 10 months ago

I have a local branch where I've tried to do this purely with the current dependency set and parsing some of the JS content with a thoughtful RegExp but it's not yet all the way there.

I just now remembered my attempt at this has actually been pushed to the enhancement/team-scouting-report branch. I don't recall exactly how much more work is required to get this ready for release (it may be as little as some test coverage?), but the fundamental underpinnings are there if you want to build off that.

If not, I will try my best to get back around to this soon.

ddogwoofwoof commented 10 months ago

@esqew this is perfect!! Thank you so much for sharing!