Closed GaspardBT closed 1 year ago
This change would require the user to be on pandas>=1.5.0
, but that isn’t specified in the package requirements, so this change should also:
pandas>=1.5.0
in the setup.py (which I’m thinking we don’t want to do)DataError
location, while still supporting the existing fallbacksAlso encountered this.
The cause of this isn't really a "breaking change in Pandas" but rather Sparkmagic reaching into Pandas' internal paths to import DataError
type.
If you look at the Panda's API reference - the correct/public import path is:
from pandas.errors import DataError
In our docker image we added a following temporary workaround which solved the issue:
sed -i 's/from pandas.core.groupby import DataError/from pandas.errors import DataError/g' /opt/conda/lib/python3.10/site-packages/autovizwidget/plotlygraphs/graphbase.py
sed -i 's/from pandas.core.groupby import DataError/from pandas.errors import DataError/g' /opt/conda/lib/python3.10/site-packages/autovizwidget/plotlygraphs/piegraph.py
+1, either running locally, or docker build and then docker run, I get the error. Sparkmagic currently (2022) is useless until some fix is done.
This has been fixed in the latest release.
Thank you all for your patience and thank you @GaspardBT for the patch!
Describe the bug Import Sparkmagic fail with `ImportError: cannot import name 'DataError' from 'pandas.core.groupby'
To Reproduce Run the Contributing steps with
python3.9.7
Then run the tests:nosetests hdijupyterutils autovizwidget sparkmagic
The tests fails with:ImportError: cannot import name 'DataError' from 'pandas.core.groupby'
Expected behavior The tests should succeed
Additional context Pandas released a new version with what can be seen as breaking changes as they change the path for errors used in sparkmagics here This can be solved by either changing the
requirements.txt
to limit pandas version under1.5.0
or to change the import path to the new ones. I created a PR that implemented the second solutions.