canot import name 'plotvol3'

slyandsmart commented 2 months ago

I have the following error. I tried to use the roboticstoolbox on my Ubuntu 22.04 Docker Container. It runs python 3.10.12 It seems to be similar to https://github.com/bdaiinstitute/spatialmath-python/issues/52

from spatialmath.base import symbolic as sym Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.10/dist-packages/spatialmath/init.py", line 3, in from spatialmath.pose2d import SO2, SE2 File "/usr/local/lib/python3.10/dist-packages/spatialmath/pose2d.py", line 26, in import spatialmath.base as smb File "/usr/local/lib/python3.10/dist-packages/spatialmath/base/init.py", line 8, in from spatialmath.base.transforms3d import * # lgtm [py/polluting-import] File "/usr/local/lib/python3.10/dist-packages/spatialmath/base/transforms3d.py", line 48, in from spatialmath.base.graphics import plotvol3, axes_logic ImportError: cannot import name 'plotvol3' from 'spatialmath.base.graphics' (/usr/local/lib/python3.10/dist-packages/spatialmath/base/graphics.py)

or i tried also

from spatialmath import base as base Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.10/dist-packages/spatialmath/init.py", line 3, in from spatialmath.pose2d import SO2, SE2 File "/usr/local/lib/python3.10/dist-packages/spatialmath/pose2d.py", line 26, in import spatialmath.base as smb File "/usr/local/lib/python3.10/dist-packages/spatialmath/base/init.py", line 8, in from spatialmath.base.transforms3d import * # lgtm [py/polluting-import] File "/usr/local/lib/python3.10/dist-packages/spatialmath/base/transforms3d.py", line 48, in from spatialmath.base.graphics import plotvol3, axes_logic ImportError: cannot import name 'plotvol3' from 'spatialmath.base.graphics' (/usr/local/lib/python3.10/dist-packages/spatialmath/base/graphics.py)

peteflorence commented 1 month ago

If not using graphics, a hacky workaround is to just not import that line, i.e. in OP's example:

vi /usr/local/lib/python3.10/dist-packages/spatialmath/base/transforms3d.py

and comment out line 48

andybarry commented 1 month ago

This is also a problem for me (running python 3.10.12)

andybarry commented 1 month ago

It looks like this is the same as #121 but the resolution there is to uninstall python3-matplotlib which is not very helpful, I need that package.

andybarry commented 1 month ago

This is a huge hack, but for the moment you can use pip install spatialmath-python-abarryfix==1.1.12 to get around this.

jbarry-bdai commented 1 month ago

@myeatman-bdai @petercorke can we get eyes on this one?

jcao-bdai commented 1 month ago

yes this seems to be the same as #121 where the conflict was

matplotlib apt installed by ROS, command dpkg - l | grep matplotlib would show version 3.5.1, it's pinned to this version;
matplotlib version required by spatialmath is a newer version (using the pip mechanism, pip install matplotlib, https://pypi.org/project/matplotlib/)

Unfortunately we don't have a good solution, only a workaround ~dpkg -r --force-depends python3-matplotlib pip install spatialmath-python # or spatialmath is already installed, do pip install matplotlib~

~EDIT: for clarify, i am replacing the line of pip install spatialmath-python with pip install matplotlib, to make the intention clear.~ ~dpkg -r --force-depends python3-matplotlib~ ~pip install matplotlib~

EDIT (08/01/2024): we're not recommending this workaround above as it is not a reliable solution (see below).

andybarry commented 1 month ago

python3-matplotlib is required by ROS and OpenCV.

Here's the mess I get if I try to uninstall it:

# sudo apt remove python3-matplotlib
[sudo] password for abarry: 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
  cppcheck evince-common fonts-lyx freeglut3 libaom-dev libarmadillo-dev libarpack2-dev libavcodec-dev libavformat-dev libavutil-dev libblas-dev libblosc-dev libcfitsio-dev libcfitsio-doc libcharls-dev
  libdav1d-dev libdc1394-dev libde265-dev libdouble-conversion-dev libevdocument3-4 libevview3-3 libexif-dev libexif-doc libfreexl-dev libfyba-dev libgdal-dev libgdcm-dev libgeos-dev libgeotiff-dev libgif-dev
  libgl2ps-dev libgl2ps1.4 libglew-dev libgphoto2-dev libheif-dev libilmbase-dev libjson-c-dev libjsoncpp-dev libkml-dev libkmlconvenience1 libkmlregionator1 libkmlxsd1 liblapack-dev liblbfgsb0 libnetcdf-c++4
  libnetcdf-cxx-legacy-dev libogdi-dev libogg-dev libopencv-calib3d-dev libopencv-contrib-dev libopencv-core-dev libopencv-dev libopencv-dnn-dev libopencv-features2d-dev libopencv-flann-dev
  libopencv-highgui-dev libopencv-imgcodecs-dev libopencv-imgproc-dev libopencv-ml-dev libopencv-objdetect-dev libopencv-photo-dev libopencv-photo4.5d libopencv-shape-dev libopencv-shape4.5d
  libopencv-stitching-dev libopencv-stitching4.5d libopencv-superres-dev libopencv-superres4.5d libopencv-video-dev libopencv-videoio-dev libopencv-videostab-dev libopencv-videostab4.5d libopencv-viz-dev
  libopencv-viz4.5d libopencv4.5-java libopencv4.5d-jni libopenexr-dev libopenjp2-7-dev libopenni-dev libopenni-sensor-pointclouds0 libopenni0 libopenni2-dev libpcl-apps1.12 libpcl-common1.12 libpcl-dev
  libpcl-features1.12 libpcl-filters1.12 libpcl-io1.12 libpcl-kdtree1.12 libpcl-keypoints1.12 libpcl-ml1.12 libpcl-octree1.12 libpcl-outofcore1.12 libpcl-people1.12 libpcl-recognition1.12
  libpcl-registration1.12 libpcl-sample-consensus1.12 libpcl-search1.12 libpcl-segmentation1.12 libpcl-stereo1.12 libpcl-surface1.12 libpcl-tracking1.12 libpcl-visualization1.12 libpoppler-dev
  libpoppler-private-dev libproj-dev libqt5designercomponents5 libqt5quicktest5 libqt5webkit5 libqt5webkit5-dev libraw1394-dev libraw1394-tools librttopo-dev libspatialite-dev libsuperlu-dev libswresample-dev
  libswscale-dev libtbb-dev libtbb12 libtheora-dev liburiparser-dev libusb-1.0-0-dev libusb-1.0-doc libutfcpp-dev libvtk9-dev libvtk9-java libvtk9-qt-dev libvtk9.1 libvtk9.1-qt libwebp-dev libx265-dev
  libxerces-c-dev libxsimd-dev mercurial mercurial-common nvidia-firmware-535-535.154.05 opencv-data openni-utils pydocstyle pyflakes3 python-matplotlib-data python3-appdirs python3-beniget
  python3-colcon-mixin python3-cycler python3-decorator python3-flake8 python3-fonttools python3-fs python3-gast python3-kiwisolver python3-lz4 python3-mccabe python3-mpi4py python3-mpmath python3-opencv
  python3-pycodestyle python3-pydocstyle python3-pydot python3-pyflakes python3-pythran python3-scipy python3-snowballstemmer python3-sympy python3-ufolib2 python3-unicodedata2 python3-vcstool python3-vcstools
  python3-vtk9 qdoc-qt5 qhelpgenerator-qt5 qt5-assistant qtattributionsscanner-qt5 qtdeclarative5-dev qtdeclarative5-dev-tools qttools5-dev qttools5-dev-tools qttools5-private-dev ros-build-essential
  ros-humble-action-tutorials-cpp ros-humble-action-tutorials-interfaces ros-humble-action-tutorials-py ros-humble-ament-cmake-auto ros-humble-ament-cmake-copyright ros-humble-ament-cmake-cppcheck
  ros-humble-ament-cmake-cpplint ros-humble-ament-cmake-flake8 ros-humble-ament-cmake-lint-cmake ros-humble-ament-cmake-pep257 ros-humble-ament-cmake-uncrustify ros-humble-ament-cmake-xmllint
  ros-humble-ament-cppcheck ros-humble-ament-cpplint ros-humble-ament-flake8 ros-humble-ament-lint-auto ros-humble-ament-lint-cmake ros-humble-ament-lint-common ros-humble-ament-pep257
  ros-humble-ament-uncrustify ros-humble-ament-xmllint ros-humble-composition ros-humble-cv-bridge ros-humble-demo-nodes-cpp ros-humble-demo-nodes-cpp-native ros-humble-demo-nodes-py
  ros-humble-depthimage-to-laserscan ros-humble-dummy-map-server ros-humble-dummy-robot-bringup ros-humble-dummy-sensors ros-humble-example-interfaces ros-humble-examples-rclcpp-minimal-action-client
  ros-humble-examples-rclcpp-minimal-action-server ros-humble-examples-rclcpp-minimal-client ros-humble-examples-rclcpp-minimal-composition ros-humble-examples-rclcpp-minimal-publisher
  ros-humble-examples-rclcpp-minimal-service ros-humble-examples-rclcpp-minimal-subscriber ros-humble-examples-rclcpp-minimal-timer ros-humble-examples-rclcpp-multithreaded-executor
  ros-humble-examples-rclpy-executors ros-humble-examples-rclpy-minimal-action-client ros-humble-examples-rclpy-minimal-action-server ros-humble-examples-rclpy-minimal-client
  ros-humble-examples-rclpy-minimal-publisher ros-humble-examples-rclpy-minimal-service ros-humble-examples-rclpy-minimal-subscriber ros-humble-geometry2 ros-humble-image-geometry ros-humble-image-tools
  ros-humble-intra-process-demo ros-humble-keyboard-handler ros-humble-launch-testing ros-humble-launch-testing-ament-cmake ros-humble-launch-testing-ros ros-humble-launch-xml ros-humble-launch-yaml
  ros-humble-lifecycle ros-humble-logging-demo ros-humble-pcl-conversions ros-humble-pcl-msgs ros-humble-pendulum-control ros-humble-pendulum-msgs ros-humble-qt-dotgraph ros-humble-qt-gui ros-humble-qt-gui-cpp
  ros-humble-qt-gui-py-common ros-humble-quality-of-service-demo-cpp ros-humble-quality-of-service-demo-py ros-humble-ros-base ros-humble-ros-core ros-humble-ros-environment ros-humble-ros2action
  ros-humble-ros2bag ros-humble-ros2cli-common-extensions ros-humble-ros2component ros-humble-ros2doctor ros-humble-ros2interface ros-humble-ros2launch ros-humble-ros2lifecycle ros-humble-ros2multicast
  ros-humble-ros2topic ros-humble-rosbag2 ros-humble-rosbag2-compression ros-humble-rosbag2-compression-zstd ros-humble-rosbag2-cpp ros-humble-rosbag2-interfaces ros-humble-rosbag2-py
  ros-humble-rosbag2-storage ros-humble-rosbag2-storage-default-plugins ros-humble-rosbag2-transport ros-humble-rosidl-default-generators ros-humble-rqt-action ros-humble-rqt-bag ros-humble-rqt-console
  ros-humble-rqt-graph ros-humble-rqt-gui ros-humble-rqt-gui-cpp ros-humble-rqt-gui-py ros-humble-rqt-image-view ros-humble-rqt-msg ros-humble-rqt-publisher ros-humble-rqt-py-common ros-humble-rqt-py-console
  ros-humble-rqt-reconfigure ros-humble-rqt-service-caller ros-humble-rqt-shell ros-humble-rqt-srv ros-humble-rqt-topic ros-humble-rttest ros-humble-shared-queues-vendor ros-humble-sqlite3-vendor
  ros-humble-sros2 ros-humble-sros2-cmake ros-humble-tango-icons-vendor ros-humble-teleop-twist-joy ros-humble-teleop-twist-keyboard ros-humble-tf2-bullet ros-humble-tf2-sensor-msgs ros-humble-tf2-tools
  ros-humble-tlsf ros-humble-tlsf-cpp ros-humble-topic-monitor ros-humble-turtlesim ros-humble-uncrustify-vendor ros-humble-zstd-vendor tango-icon-theme tcl-dev tcl8.6-dev tk-dev tk8.6-dev uncrustify
  unicode-data vtk9
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  python3-matplotlib ros-humble-desktop ros-humble-rqt-bag-plugins ros-humble-rqt-common-plugins ros-humble-rqt-plot
0 upgraded, 0 newly installed, 5 to remove and 106 not upgraded.
After this operation, 19.7 MB disk space will be freed.
Do you want to continue? [Y/n]

We can't ask people to uninstall OpenCV and ROS to use this, is there any other workaround?

andybarry commented 1 month ago

sudo dpkg -r --force-depends python3-matplotlib does work for me, but I think the package should be able to be used by default and a lot of people will run into this.

jcao-bdai commented 1 month ago

@slyandsmart @andybarry i edited my "workaround" in previous reply to clarify that we still install matplotlib, but using pip install mechanism instead. but again, that's a hack, not a solution.

~please only try it if you want to participate in troubleshooting this issue as it might break your workflow and cost you more time.~

~for a proper solution, we can look through the spatialmath code and perhaps we don't have to use features from newer matplotlib versions, in which case we will push an update and stick to the same matplotlib version pinned by ROS2 humble.~

EDIT (08/01/2024): we're not recommending this workaround above as it is not a reliable solution (see below).

myeatman-bdai commented 1 month ago

Would the best solution be moving all plotting functions to their own file that doesn't get imported automatically by anything?

The plotting functionality might be unusable depending on your version of matplotlib, but everything should still work?

andybarry commented 1 month ago

for a proper solution, we can look through the spatialmath code and perhaps we don't have to use features from newer matplotlib versions, in which case we will push an update and stick to the same matplotlib version pinned by ROS2 humble.

That would be fantastic!

andybarry commented 1 month ago

This issue continues to be a big problem for us. The apt hack breaks apt for everything else, so it makes all sorts of things not work.

To be honest, we're on the verge of using something else because of this issue.

jcao-bdai commented 1 month ago

@andybarry sorry to hear that! I am escalating this issue and hopefully cut a new release with a properly pinned matplotlib version. will update here again once we have an estimate.

jcao-bdai commented 1 month ago

@andybarry it seems simply pinning the version might work, https://github.com/bdaiinstitute/spatialmath-python/pull/133 but we still need to run more tests to make sure this change doesn't break anything.

as the ROS2's apt-installed matplotlib (python3-matplotlib) is slightly old, it would also prevent installing numpy 2.0 or newer - we had to modify the pyproject.toml to narrow the numpy version to "numpy ... <2".
tests we've done so far are listed in the PR description and there are a few failure cases we need to figure out.

andybarry commented 1 month ago

This is awesome, I will test it tomorrow!

On Thu, Aug 1, 2024, 9:55 AM Jien Cao @.***> wrote:

@andybarry https://github.com/andybarry it seems simply pinning the version would work, #133 https://github.com/bdaiinstitute/spatialmath-python/pull/133

We still need to run more tests to make sure this change doesn't break anything. In particular, as the ROS2's apt-installed matplotlib (python3-matplotlib) is slightly old, it would also prevent installing numpy 2.0 or newer - we had to modify the pyproject.toml to narrow the numpy version to "numpy ... <2".

Tests we've done so far are listed in the PR description.

If you would like to participate in the testing, please feel free to checkout the test branch issue-127/pin-matplotlib-version and pip install ..

— Reply to this email directly, view it on GitHub https://github.com/bdaiinstitute/spatialmath-python/issues/127#issuecomment-2263133968, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ5LKTI4TIFVK3GM5IQSJLZPI43ZAVCNFSM6AAAAABKLRSMPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRTGEZTGOJWHA . You are receiving this because you were mentioned.Message ID: @.***>

jcao-bdai commented 1 month ago

@andybarry please kindly share your findings.

When testing in a ROS2 docker (humble, community version), i noticed that scipy also came preinstalled with it (apt installed), and that version was also a bit old, which would further narrow the range of numpy to <=1.25 (change not included in this work-in-progress PR), seemingly a bit too restrictive.

I am curious if you see similar issue (scipy throws warning requiring lower version of numpy) in your setup?

andybarry commented 1 month ago

This appears to work for me! We will try it on a few different configurations here and let you know if we find any issues.

Yazan-Soliman commented 4 weeks ago

I also want to report that this is also working for me. ROS2 humble Ubuntu 22.04

jcao-bdai commented 4 weeks ago

@Yazan-Soliman thank you for testing the changes! Did you have to change the numpy version range (in current PR, it's numpy>=1.22,<2) in your setup?

Asking because in my test the ROS2 docker seemed to have an old scipy which warned me to further narrow the numpy range, but perhaps the docker image i used was not typical.

Yazan-Soliman commented 4 weeks ago

@jcao-bdai in my environment i am using numpy Version: 1.24.4

bdaiinstitute / spatialmath-python

canot import name 'plotvol3' #127