No module named 'encodings' with python 3.7 on OSX

[pyAstroDude]

Hi, I am new here so not sure if this is the place where I ask advise about installation issues. Also, I did not find a slack channel for this tutorial.

Note that I am working on a mac laptop OS X 10.13.6. While following your instructions in INSTALL.md I get an error when I run instruction # 6, setting a new environment.

The error is as follows: Executing transaction: / b"Fatal Python error: initfsencoding: unable to load the file system codec\nModuleNotFoundError: No module named 'encodings'\n\nCurrent thread 0x00007fff9a72c380 (most recent call first):\n" ERROR conda.core.link:_execute_post_link_actions(658): An error occurred while installing package 'conda-forge::widgetsnbextension-3.4.2-py37_1000'. LinkError: post-link script failed for package conda-forge::widgetsnbextension-3.4.2-py37_1000 running your command again with -v will provide additional information location of failed script: /Users/sshenoy/Applications/miniconda3/envs/satpy/bin/.widgetsnbextension-post-link.sh ==> script messages <== Fatal Python error: initfsencoding: unable to load the file system codec ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007fff9a72c380 (most recent call first):

Attempting to roll back.

failed ERROR conda.core.link:_execute(568): An error occurred while installing package 'conda-forge::widgetsnbextension-3.4.2-py37_1000'. LinkError: post-link script failed for package conda-forge::widgetsnbextension-3.4.2-py37_1000 running your command again with -v will provide additional information location of failed script: /Users/sshenoy/Applications/miniconda3/envs/satpy/bin/.widgetsnbextension-post-link.sh ==> script messages <== Fatal Python error: initfsencoding: unable to load the file system codec ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007fff9a72c380 (most recent call first):

Attempting to roll back.

Rolling back transaction: done

LinkError: post-link script failed for package conda-forge::widgetsnbextension-3.4.2-py37_1000 running your command again with -v will provide additional information location of failed script: /Users/sshenoy/Applications/miniconda3/envs/satpy/bin/.widgetsnbextension-post-link.sh ==> script messages <== Fatal Python error: initfsencoding: unable to load the file system codec ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007fff9a72c380 (most recent call first):

Not sure what is going on here. Any help will be appreciated.

[mraspaud]

@pyAstroDude Welcome ! You can join us on the regular pytroll slack to discuss this issue if you want, but this looks like to be something deeper that what pytroll/satpy is responsible for. I would advise reporting this as a conda issue instead.

[djhoese]

@pyAstroDude As @mraspaud says, this seems to be something wrong with the Anaconda distribution and/or your system.

Did you installed Anaconda/Miniconda from a fresh install or are you using a version you had installed previously?

[pyAstroDude]

@mraspaud and @djhoese, Thanks. I have a previously installed version of miniconda. I also found out that the issue lies in python=3.7, if I run python=3.6 then I do not get this error. So, my questions now is: Do I need python 3.7 to complete this tutorial or will python 3.6 is OK?

[djhoese]

Python 3.6 should be fine. I'm a little worried that others are going to run in to this issue. Is there an issue or stackoverflow question that you found that talks about this? Most importantly I'm wondering if this is only an issue for older versions of anaconda/miniconda or if this is a known issue for python 3.7 in general.

[pyAstroDude]

@djhoese I don't consider myself an expert in this but I found the issue of 3.6 versus 3.7 by trial an error. I did try to search stackoverflow/online but did not get too far.

[djhoese]

Ok, thanks for the info. If you have the time to try a few things for me that would be great so we can narrow this issue down for anyone else experiencing it. I have to admit I wrote the instructions to use python 3.7 but was actually testing things with python 3.6. I'm testing things on my own OSX laptop with python 3.7 now.

I was going to request, since you have an older miniconda installation, if you could update everything in your base environment conda update --all and see if the create command (with py37) works then. Thanks again for any help you can provide, I will see if I can reproduce the same issues.

[pyAstroDude]

Sure no problem. I will try it and let you know.

[pyAstroDude]

OK @djhoese I did the test you asked for, I did conda update --all and then tried creating the satpy environment with python 3.7 ........ same errro as before :(

DO you think it is some thing to do with python 3.7 on Macs, may be?

[djhoese]

I just ran the installation instructions on a recently updated conda environment (conda update --all) and the conda create command succeeded:

conda create -y -n satpy_tutorial python=3.7 satpy rasterio imageio imageio-ffmpeg cartopy geoviews notebook requests tqdm

NOTE: I named my environment satpy_tutorial because I already had the satpy environment.

Preparing transaction: done
Verifying transaction: done
Executing transaction: \ b'Enabling notebook extension jupyter-js-widgets/extension...\n      - Validating: \x1b[32mOK\x1b[0m\n'
done
#
# To activate this environment, use
#
#     $ conda activate satpy_tutorial
#
# To deactivate an active environment, use
#
#     $ conda deactivate

When you run conda create it should be using the conda from your installed anaconda/miniconda environment and shouldn't be using any outside python environments. I am running this on OSX 10.14.5. Please be sure you don't have a different python-like environment activated or being used on your system.

If you are still having issues you may have to reinstall miniconda, following the instructions to add the proper scripts to your .bashrc file, and make sure you no longer have the old miniconda. For the Satpy tutorial at SciPy 2019 it should be fine to have Python 3.6, but I'm worried you may have issues for other tutorials or other work in the future so it may be worth trying to start fresh in the long run.

If someone else has this issue I will definitely try running from a brand new anaconda/miniconda environment and see what I get. I'll also check in with the conda-forge community to see if this sounds familiar to any of them.

[djhoese]

@pyAstroDude Could you double check what version of conda you have?

FYI, although it won't fix the issue you had, I did just make some fixes to the data download scripts so please update your copy of this repository (git pull) to make sure you have all the data.

[pyAstroDude]

conda --version conda 4.6.14

Thanks I will update my repo.

[pyAstroDude]

@djhoese I update conda to 4.7.5 and still have the same issue i.e can create environment with py36 but not py37.

Also after I updated conda and activated the satpy environment, I got an error when I tried to download the data. Here is the error

Downloading lookup tables used by pyspectral... Traceback (most recent call last): File "download_data.py", line 100, in <module> sys.exit(main()) File "download_data.py", line 86, in main ret &= download_pyspectral_luts() File "download_data.py", line 13, in download_pyspectral_luts from pyspectral.utils import download_luts, download_rsr ModuleNotFoundError: No module named 'pyspectral'

When I do: conda list pyspectral

I get the following: # packages in environment at /Users/sshenoy/Applications/miniconda3/envs/satpy36: # # Name Version Build Channel pyspectral 0.8.9 py_0 conda-forge

[djhoese]

So you are saying you create a python 3.6 environment by following the tutorial instructions (which include installing pyspectral), conda says pyspectral is installed, but when you try to import it (via the download script) it says it doesn't exist?

Are you sure you activated your environment properly? Do you have PYTHONPATH set?

What happens if, with the activated python environment, you do python -c "import pyspectral"?

Edit: Have you tried completely deleting/removing the conda environments you created and creating new ones?

[pyAstroDude]

So you are saying you create a python 3.6 environment by following the tutorial instructions (which include installing pyspectral), conda says pyspectral is installed, but when you try to import it (via the download script) it says it doesn't exist?

Yes.

Are you sure you activated your environment properly? Do you have PYTHONPATH set?

I did conda activate satpy36

Yes I have PYTHONPATH set to /Users/sshenoy/dps/pipelines/hawc/pipeline:/Users/sshenoy/SDS_PyAstro:/Users/sshenoy/dps:/Users/sshenoy/dps/pipelines/pypecal/pipecal/hawc

What happens if, with the activated python environment, you do python -c "import pyspectral"?

Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'pyspectral'

Edit: Have you tried completely deleting/removing the conda environments you created and creating new ones?

No. not yet. I have lot of other packages that I need for work so I am a bit hesitant to do it right now. but will create a new installation of miniconda and try it again

[djhoese]

I would unset PYTHONPATH. I think it is conflicting with conda create and your normal use of python from conda environments.

[djhoese]

I'm going to close this for now. If you still have issues we can reopen this, but it sounds like this is a problem with your environment specifically.

[pyAstroDude]

@djhoese Thanks so much for the help. The issue was with conda update and some other other environment I had. Anyway, I reinstalled every thing from scratch and have got everything working. Thanks again.