Closed miguelgfierro closed 6 months ago
@loomlike it seems we only use pandera when creating a fake movielens dataset, do you know why we are using that dependency and if we can drop it?
This also affects Python 3.8.
FYI @SimonYansenZhao @anargyri more problems with dependencies
But maybe the typing
dependency is the problem.
@anargyri the error @miguelgfierro is sharing is from Python 3.7.17 so technically it is >= 3.7.
Yes, tracing the error goes back to the typing thing. there is a stalk overflow post about two solutions to this problem. https://stackoverflow.com/questions/77247446/cannot-import-name-self-from-typing
Thanks, so pandera
needs to handle this. We need to remove the dependency then.
This is the original commit that introduced pandera
btw https://github.com/recommenders-team/recommenders/commit/fd33efe2d6fc1f736809665c8d5657476374de32
from recommenders.datasets import movielens
I could only replicate the issue in 3.7, not in 3.8. @john0isaac what machine did you use? mine is a Standard_E4ds_v4 (4 cores, 32 GB RAM, 150 GB disk). What did you do to get this error? I tried
conda create -n foo python=3.8
conda activate foo
pip install recommenders
python
Python 3.8.18 (default, Sep 11 2023, 13:40:15)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from recommenders.datasets import movielens
>>>
@anargyri
Thanks @john0isaac I don't have quota for this type of machine, I am afraid. I should make a commit for Python 3.8 too. What happens if you try with higher Python versions on that machine?
@anargyri It's serverless doesn't consume lots of money and enables the spark jobs don't worry about quota.
As for your question as this is a preconfigured compute there are only two options python 3.8 or python 3.10 and python 3.10 is not supported by the recommenders package so, you need to fix this in order for this environment to work on Azure ML.
I am fixing this in a new PR. Until we release the fix on PyPI, one way to bypass the issue is after you have done pip install recommenders
you do a pip install "pandera[strategies]>=0.6.5,<0.18"
This should uninstall the latest version of pandera (which is causing the error) and install the previous one.
Thanks
Description
In python 3.7, movielens doesn't load due to a conflict with Pandera library:
In which platform does it happen?
How do we replicate the issue?
Pandera recently release 0.18.0 see https://pypi.org/project/pandera/#history
With Pandera < 0.18.0, the code works.
Expected behavior (i.e. solution)
Other Comments