issue about mini-conda install wrf-python

[LeiNate-ZHU]

Hello, I am just using mini-conda to install the wrf-python package. While I tested if it can work, I used the code and got error like " File "./test.py", line 5, in from netCDF4 import Dataset File "/home1/03716/tg830159/.local/lib/python2.7/site-packages/netCDF4/init.py", line 3, in from ._netCDF4 import * ImportError: /home1/03716/tg830159/.local/lib/python2.7/site-packages/netCDF4/_netCDF4.so: undefined symbol: PyUnicodeUCS2_DecodeASCII" While if I used a python version that I install before, no netcdf error but can not use the wrf-python package. Anyone can give me some idea?

"#!/usr/bin/env python

from future import print_function from netCDF4 import Dataset from wrf import getvar

diri = './' ncfile = Dataset(diri + "wrfout_d01_2015-09-28_12:00:00")

wrfout_d01_2015-09-28_12:00:00

Get the Sea Level Pressure

slp = getvar(ncfile, "slp") print(slp) "

[bladwig1]

I suspect you have an environment problem. Based on the error message, that doesn't look like it's trying to use the netCDF4 package from a miniconda installation. Did you really install miniconda to a directory called ".local"? By default, miniconda2 is installed in ~/miniconda2. You can run the command "which python" to see which interpreter is being used.

In any case, the problem is with the netCDF4 installation, not with wrf-python. I would recommend reinstalling miniconda in the default location. Set your path to point to ~/miniconda2/bin. Then run:

conda create -n my_env -c conda-forge wrf-python netcdf4 source activate my_env

(run your tests)

source deactivate.

Hope this helps.

[LeiNate-ZHU]

Thanks. I also think the problem should be from the netcdf. I installed the miniconda as the default location just like what you said, but I don't know why the netcdf still point to the ./local directory even I have done with your advice again. I have set the path like export PATH="/home1/03716/tg830159/miniconda2/bin:$PATH".

Hope you can give me some further help, thanks very much.

Best, Lei

[bladwig1]

When you type the command "which python", what does it report? If it doesn't report the miniconda installation, then you have a environment problem. Also, make sure you don't have anything in a PYTHONPATH environment variable. After you run "source activate test_env", most of these problems should go away. Also, see what the result of the "env" command reports back.

[LeiNate-ZHU]

When I type "which python", it shows "~/miniconda2/bin/python". Also, I do not set the PYTHONPATH in my environment variable. As i am using the TACC stampede supercomputer, I think it might already set the PYTHONPATH by default, does it mean that I should change something different? When I type "source activate test_env", it shows "Traceback (most recent call last): File "/home1/03716/tg830159/miniconda2/bin/conda", line 6, in sys.exit(conda.cli.main()) File "/home1/03716/tg830159/miniconda2/lib/python2.7/site-packages/conda/cli/main.py", line 154, in main activate.main() File "/home1/03716/tg830159/miniconda2/lib/python2.7/site-packages/conda/cli/activate.py", line 160, in main prefix = prefix_from_arg(sys.argv[3], shelldict=shelldict) File "/home1/03716/tg830159/miniconda2/lib/python2.7/site-packages/conda/cli/activate.py", line 60, in prefix_from_arg prefix = locate_prefix_by_name(context, arg.replace('/', os.path.sep)) File "/home1/03716/tg830159/miniconda2/lib/python2.7/site-packages/conda/base/context.py", line 557, in locate_prefix_by_name raise CondaEnvironmentNotFoundError(name) conda.exceptions.CondaEnvironmentNotFoundError: Could not find environment: test_env . You can list all discoverable environments with conda info --envs."

Sorry about that I am asking the question which should be handled by myself.

Best, Lei

[bladwig1]

Did you create the environment first with:

conda create -n test_env -c conda-forge wrf-python netcdf4

Also, on our supercomputer here, we use a "module load" system, that will set numerous environment variables as well. You should be able to see what is being set with the "env" command.

It's no problem to ask for installation support, but after we get this resolved, the wrfpython-talk email list is a better place for general support (I just approved you). I want to try to keep github for bug related issues. For now though, let's try to get this working for you.

[LeiNate-ZHU]

Yes, as you advised before. I used the commend "conda create -n my_env -c conda-forge wrf-python netcdf4" to install netcdf4. Also, I can use the commend "source activate my_env". The supercomputer we also use "module load" to get some variable. And I load netcdf and python my my bashrc. I typed "env", the log shows in the link: http://hfip.psu.edu/luz30/wrf_python.log

Thanks. Lei

[bladwig1]

Ok, don't load python or netcdf from the "module load" system if you are using miniconda. Miniconda ships with its own python interpreter and netcdf libraries, and it's probably conflicting with the one on the system.

[It looks to me like your HPC python was built with Intel, so you may want to consider building wrf-python from source if you want one that plays nicely with your Intel version of python (or see if a system admin can build and install it as a module). It requires a C and Fortran compiler, which you should have. ]

Before doing all that, see if you can isolate miniconda by not loading any modules, restart your shell, create and activate the conda environment with the commands above. In any case, you definitely don't want to load the HPC python module if you are using miniconda.

[LeiNate-ZHU]

Sorry about that it still shows the same problem: " from netCDF4 import Dataset File "/home1/03716/tg830159/.local/lib/python2.7/site-packages/netCDF4/init.py", line 3, in from ._netCDF4 import * ImportError: libnetcdf.so.7: cannot open shared object file: No such file or directory"

[bladwig1]

Somehow a netcdf4-python library got installed in ~/.local, which can happen if you did a "pip install --user " to install it. Let's try moving that ~/.local directory and hopefully miniconda2 will stop finding it. (I don't know why that's happening). Try doing "mv ~/.local to ~/.foo". Or, if you don't need anything in there, you could just delete it.

Let's also make sure you aren't running some other 'conda' command that might be lurking in a path.

Run 'which conda' and 'which python' and make sure both point to ~/miniconda2/bin.

[LeiNate-ZHU]

Finally, it works now after I moved the ./local directory to other place. I am really grateful for your patient and prompt responses. Best wishes, Lei

[bladwig1]

Awesome! I don't know why miniconda2 was picking up that .local directory, but it might be a system related thing. In any case, I should have had you move that directory first.

I'm going to close this issue. If you have any more issues, you can now write to wrfpython-talk@ucar.edu or use the google group directly.

[saguiMet]

Hello. I installed wrf-python using the command "conda create -n my_env -c conda-forge wrf-python". I used the code and got error like: "ImportError: No module named wrf".

Any idea?

[erogluorhan]

Hello. I installed wrf-python using the command "conda create -n my_env -c conda-forge wrf-python". I used the code and got error like: "ImportError: No module named wrf".

Any idea?

After your installation of conda environment (i.e. my_env), did you activate this newly created environment by calling the command:

conda activate my_env

[saguiMet]

Hello. I installed wrf-python using the command "conda create -n my_env -c conda-forge wrf-python". I used the code and got error like: "ImportError: No module named wrf". Any idea?

After your installation of conda environment (i.e. my_env), did you activate this newly created environment by calling the command:

conda activate my_env

Yes i did! I've got other environmet with different 'my env' name. This is an environment conflict?

[erogluorhan]

Yes i did! I've got other environmet with different 'my env' name. This is an environment conflict?

No, it shouldn't as one is "my_env" and the other is "my env". Could you confirm you can see wrf-python package as installed when you call:

conda list