Login Failing with utils.py

[spaceman-spiff-else]

Kept receiving the error "mechanicalsoup.utils.LinkNotFoundError" when following the login instructions on your page using my Kenpom credentials. I noticed I was able to fix it when I commented out these two lines in utils.py:

8 from ._DESAdapter import DESAdapter, environment_requires_DES_adapter 24 if environment_requires_DES_adapter(): 25 session.mount('https://kenpom.com/', DESAdapter())

Hope this helps!

[j-andrews7]

See #44, this is probably related to #33 and the fix for it. Updating to python 3.11 might fix it, but we need to look into this further.

[esqew]

Hi all, sorry to be late to the party (again, thanks to real life).

@spaceman-spiff-else Can you comment with info regarding your execution environment (OS, Python version) as well as the full stack trace of the error itself?

Reviewing recently-raised issues (that I missed thanks to my day job being a day-and-night job in the new year) it seems this isn't working as expected for everyone. The LinkNotFoundErrors being somehow related to DESAdapter are interesting as the configuration that it does for older Python versions shouldn't directly impact the content that comes back to the point where it throws these types of errors. For that code, I'd expect something more along the lines of the error described in #44 about the ciphers and whatnot.

[spaceman-spiff-else]

Hey @esqew - no worries. Definitely higher priorities out there than troubleshooting this. Thanks for getting back to me though. I'm running Python 3.10.1 on Windows. For some reason I'm not able to create the error again. Attached screenshot is the current error I get (when I revert back to the full utils.py code). I was; however, able to pull the old error from a log. See below: File "C:\Users\XXX\AppData\Local\Programs\Python\Python310\lib\site-packages\mechanicalsoup\stateful_browser.py", line 222, in select_form raise LinkNotFoundError() mechanicalsoup.utils.LinkNotFoundError

I'll upload to python 11 and revert back on any changes.

[spaceman-spiff-else]

Confirmed - no changes with python 11

[nickostendorf]

@spaceman-spiff-else Did you install kenpompy using pip or did you download the zip from Github and install using pip install .? I'm wondering if the relative import error is because of how the package was configured.

The LinkNotFoundError can be a slew of things, but I believe this is getting thrown because kenpom.com is rejecting traffic due to mechanicalsoup not using the correct ciphers (DESAdapter not applying them to the mechanicalsoup session)? I could be wrong though.

As @esqew pointed out, this still should not be getting throw if you have Python >= 3.11.

@spaceman-spiff-else can you run your script again and then run python -V and paste the terminal output here

[esqew]

I cannot reproduce this using either Python 3.9.6 or Python 3.11.0, albeit my environment is macOS. I'll have to finagle a Windows test environment sometime soon to see if I can reproduce there.

[spaceman-spiff-else]

Here's the requested screenshot. Run using Windows Command Prompt. Again, just commenting out _from .DESAdapter import DESAdapter works fine and I haven't found online a package of this name and so have never been able to download to my python environment. Also, kenpompy was installed using pip.

[esqew]

@spaceman-spiff-else Can you try invoking schedules.py using the Python interpreter, i.e. py schedules.py and let us know if you see the same issue? I'm curious if invoking it the way you are may be contributing to the issue you're seeing (I personally have never been able to get Python on Windows to execute Python files by themselves without invoking the interpreter itself).

[j-andrews7]

As an aside, I highly recommend you look into the Windows Subsystem for Linux if you plan on trying to do any real programming on Windows @spaceman-spiff-else. It will save you much pain in the long run.

[esqew]

I missed this in my first read-through and wanted to address it for absolute clarity:

I haven't found online a package of this name and so have never been able to download to my python environment.

_DESAdapter is a something we developed for #38 to fix some issues which were contributing to Cloudflare blocking our scrapes. It is not a discrete package and is not something you need to download separately since the complete code for the class is included in the codebase here.

[spaceman-spiff-else]

Got it thanks for the info @j-andrews7. What would you say is the benefit to of WSL? I've just been lazy with using Command Prompt. I did finally move my script to a Jupyter Notebook and unfortunately still received the same error.

@esqew - same thing happens when invoking the interpreter. Honestly I don't know enough about python to understand the difference between all the ways to execute my script. I just know it only works in Command Prompt without the DESAdapter import.

[j-andrews7]

Got it thanks for the info @j-andrews7. What would you say is the benefit to of WSL? I've just been lazy with using Command Prompt. I did finally move my script to a Jupyter Notebook and unfortunately still received the same error.

Having stuff just work, mostly. Pretty much every programming tool/library/package/whatever in existence works with Linux. Getting stuff to work natively in Windows is often a headache.

But WSL works great, and I've been using it professionally for years to great effect. Keep the convenience of Windows while still having a true unix command-line/environment. It's like cygwin, except it doesn't suck.

Trust me, trying to stick solely with native Windows and programming is a path that leads only to pain.

[spaceman-spiff-else]

@j-andrews7 good to know! I'm sure I've already run into countless errors that could have been simply a Windows environment issue. I'll definitely give WSL a try!

[esqew]

I cannot reproduce this issue on any platform/Python version combination.

I'll do what I can to resolve the relative import issue in the next push that goes to master, but unless we can reliably reproduce the other symptoms relating to the login failure I'll consider this closed for the time being.