pysmu throws ImportError on fresh Ubuntu (18.04 and 20.04)

[cyber-g]

Calling pysmu on fresh Ubuntu (18.04 and 20.04) causes an error and aborts the execution:

Traceback (most recent call last):
  File "/usr/bin/pysmu", line 24, in <module>
    from urllib import urlretrieve
ImportError: cannot import name 'urlretrieve'

The error is illustrated by this simple Dockerfile: Dockerfile.zip

The solution is well documented on the internet:

In Python 3.x, the urlretrieve function is located in the urllib.request module.

Source: https://stackoverflow.com/a/21171861

[cyber-g]

Replace line 24 by: from urllib.request import urlretrieve

fixes the initial issue.

But a new error is thrown:

Traceback (most recent call last):
  File "/usr/bin/pysmu", line 25, in <module>
    import urllib2
ModuleNotFoundError: No module named 'urllib2'

[cyber-g]

urllib2 cannot be used for python3. A topic on StackOverflow talking about that

Removing the import fixes the issue but I do not know if this causes invisible side effects.

[cristi-iacob]

Hi @cyber-g !

I tried to reproduce this issue on a clean Ubuntu 20 image, but it all seems to work good for me. I tried both installing libsmu and pysmu with the .deb packages from the latest release and building them manually and both methods worked as expected. I will take a look on your Docker file as well.

I will try to reproduce this issue as well in order to understand where it comes from and if your suggested solution causes other side effects. In the meanwhile, if something doesn't work with your version, you might try installing libsmu/pysmu using the .deb packages from the latest Release and see if this solves the issue for you.

Thank you!

[cyber-g]

Dear @cristi-iacob , Thanks for your reply. I have tried the installation only on simple Docker containers. (I have not tried on GUI VM)

You will see, my Dockerfile is the most simple possible and I just replace all "18" by "20" to switch between 18.04 and 20.04.

In the Dockerfile, I download the .deb packages and apt install them. Installation is ok ; the error pops only at the execution of libsmu/bindings/python/bin/pysmu

Let me know if you want me to try further tests. I also would be glad to help on any python feature. I am not a very experienced programmer but I may help on simple features.

You may also be able to answer one of my an ancillary question: I could not assert if the python bindings are for python2 or python3 ; it seems to be written for both. But, as shown in my previous comments, some python2 modules are not available in python3 ; and I was expecting some "switches" in pysmu for the appropriate imports. Do you have more information about the python part of the project ?

[cristi-iacob]

I understand now. Initially I thought that the issue appears when importing pysmu in a Python script. But after your last message I figured it out that you were trying to use the pysmu CLI.

About the bug: you are right, the issue is the one described in the posts you presented in the first comments. The problem is that urllib changed its API from Python2 to Python3 and the version from pysmu is the one for Python2. The solution for this issue would be something like this:

# Try importing urlib for Python3. If the script is run using Python2, the fallback imports will be used.
try: 
    from urllib.request import urlretrieve
except ImportError:
    from urllib import urlretrieve
    from urllib2 import urlopen

About the libsmu Python bindings: pysmu is the resulted package after compiling libsmu with the Python bindings. You can find more details about its implementation in its main script. As you can see, in this file there are "switches" for appropriate imports, based on the Python version you are using. You can choose which Python version is used for pysmu when building the libsmu library manually, using the USE_PYTHON2 CMake option. You can read more about the CMake option in the README page. If USE_PYTHON is set on 1 (which is also its default value), Python2 will be used. Otherwise Python3 will be used. The resulting pysmu from the build process is Python version specific (i.e. it will work only for the desired Python version).

I think your confusion came from the fact that both the module and the CLI (libsmu/bindings/python/bin/pysmu) have the same name. You can use the CLI for its specific defined methods (calibration, firmware update etc.), but if you want to use pysmu for further development, the pysmu module is the one you need (the one that is built and which you can import in a Python script). As already said, the pysmu module contains switches for the libraries versions, based on the Python version you are using. Since most of the people are using the module and not the Python CLI, this issue existed for quite a while and we didn't notice it. Thanks for pointing it out!

About the .deb packages: we only released the packages for Python3 because Python2 is out of support since the 1st of January 2020. Anyway, pysmu further supports Python2 if manually built, so the CLI should contain such "switches" for the dependencies imports.

Please let me know if you have any further questions or if I didn't explain something clear enough! Thank you!

[cyber-g]

Many thanks for your explanations; everything is clear