import and installation error

[syedhamidali]

Dear @rcjackson, I want to say that I was trying to install PyDDA, and it got installed successfully, but I cannot import it. It throws some errors, as listed below. I manually tried to install jaxlib=0.3.7 using conda, and somehow it doesn't install it.

the error is

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In [1], line 4
      2 warnings.filterwarnings("ignore")
      3 import pyart
----> 4 import pydda
      5 get_ipython().run_line_magic('matplotlib', 'inline')

File ~/miniconda3/envs/dda/lib/python3.8/site-packages/pydda/__init__.py:7
      1 """
      2 ========================================
      3 PyDDA: Pythonic Direct Data Assimilation
      4 ========================================
      5 """
----> 7 from . import cost_functions
      8 from . import retrieval
      9 from . import vis

File ~/miniconda3/envs/dda/lib/python3.8/site-packages/pydda/cost_functions/__init__.py:94
     92 from ._cost_functions_numpy import calculate_model_gradient
     93 from ._cost_functions_numpy import calculate_point_cost, calculate_point_gradient
---> 94 from .cost_functions import J_function, grad_J

File ~/miniconda3/envs/dda/lib/python3.8/site-packages/pydda/cost_functions/cost_functions.py:10
      7     TENSORFLOW_AVAILABLE = False
      9 try:
---> 10     from jax.config import config
     11     config.update("jax_enable_x64", True)
     12     from jax import float0

File ~/miniconda3/envs/dda/lib/python3.8/site-packages/jax/__init__.py:35
     30 del _cloud_tpu_init
     32 # Confusingly there are two things named "config": the module and the class.
     33 # We want the exported object to be the class, so we first import the module
     34 # to make sure a later import doesn't overwrite the class.
---> 35 from jax import config as _config_module
     36 del _config_module
     38 from jax._src.config import (
     39   config as config,
     40   enable_checks as enable_checks,
   (...)
     57   transfer_guard_device_to_host as transfer_guard_device_to_host,
     58 )

File ~/miniconda3/envs/dda/lib/python3.8/site-packages/jax/config.py:17
      1 # Copyright 2018 Google LLC
      2 #
      3 # Licensed under the Apache License, Version 2.0 (the "License");
   (...)
     14 
     15 # TODO(phawkins): fix users of this alias and delete this file.
---> 17 from jax._src.config import config

File ~/miniconda3/envs/dda/lib/python3.8/site-packages/jax/_src/config.py:29
     25 import warnings
     27 from absl import logging
---> 29 from jax._src import lib
     30 from jax._src.lib import jax_jit
     31 from jax._src.lib import transfer_guard_lib

File ~/miniconda3/envs/dda/lib/python3.8/site-packages/jax/_src/lib/__init__.py:94
     91   return _jaxlib_version
     93 version_str = jaxlib.version.__version__
---> 94 version = check_jaxlib_version(
     95   jax_version=jax.version.__version__,
     96   jaxlib_version=jaxlib.version.__version__,
     97   minimum_jaxlib_version=jax.version._minimum_jaxlib_version)
    101 # Before importing any C compiled modules from jaxlib, first import the CPU
    102 # feature guard module to verify that jaxlib was compiled in a way that only
    103 # uses instructions that are present on this machine.
    104 import jaxlib.cpu_feature_guard as cpu_feature_guard

File ~/miniconda3/envs/dda/lib/python3.8/site-packages/jax/_src/lib/__init__.py:83, in check_jaxlib_version(jax_version, jaxlib_version, minimum_jaxlib_version)
     80 if _jaxlib_version < _minimum_jaxlib_version:
     81   msg = (f'jaxlib is version {jaxlib_version}, but this version '
     82          f'of jax requires version >= {minimum_jaxlib_version}.')
---> 83   raise RuntimeError(msg)
     85 if _jaxlib_version > _jax_version:
     86   msg = (f'jaxlib version {jaxlib_version} is newer than and '
     87          f'incompatible with jax version {jax_version}. Please '
     88          'update your jax and/or jaxlib packages.')

RuntimeError: jaxlib is version 0.1.75, but this version of jax requires version >= 0.3.7.

and the status of the manual installation of jaxlib is

(dda) syed44@cos-7dawso-1608 ~ % conda install jaxlib=0.3.7 -c conda-forge
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: |

[rcjackson]

That version of jaxlib may not be compatible with Python 3.8. I would try to uninstall jaxlib to see if you can get it to import. If you are not using Jax for the engine then PyDDA will work without it. It should recognize the lack of Jax and act accordingly.

[syedhamidali]

Hi @rcjackson, I would like to know how I can use it as it is meant to. Which python version should I use? Should I install JAX, should tensor flow be 2.6 only, or the latest ones should also work? conda create -n dda python=3.10 tensorflow tensorflow-probability jax arm_pyart jupyter -c conda-forge How about this command?

[rcjackson]

Hamid,

Jax and TensorFlow are optional requirements for PyDDA so you don’t need either to run it. However, if you wish to run it with TensorFlow which is my recommendation, you need to install the proper TensorFlow for your version of Python. If you removed jax from that command you should be able to have matching tensorflow and tensorflow-probability, as the TensorFlow you need depends on the Python version and the version of tensorflow-probability you have. Conda should match up the needed versions together if you do that command minus jax.

Bobby

From: Hamid Ali Syed @.> Date: Monday, November 21, 2022 at 2:57 PM To: openradar/PyDDA @.> Cc: Jackson, Robert @.>, Mention @.> Subject: Re: [openradar/PyDDA] import and installation error (Issue #76)

Hi @rcjacksonhttps://github.com/rcjackson, I would like to know how I can use it as it is meant to. Which python version should I use? Should I install JAX, should tensor flow be 2.6 only, or the latest ones should also work? conda create -n dda python=3.10 tensorflow tensorflow-probability jax arm_pyart jupyter -c conda-forge How about this command?

— Reply to this email directly, view it on GitHubhttps://github.com/openradar/PyDDA/issues/76#issuecomment-1322631078, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFIQA5DXUBJFYOS64G53NELWJPO35ANCNFSM6AAAAAASG6BNL4. You are receiving this because you were mentioned.Message ID: @.***>

[syedhamidali]

Dear Bobby, Thank you for this information. I would also like to know a few more things, as listed below;

1. to perform a multi-doppler analysis, you are using the readily transformed gridded data in the examples. Should I do gridding to radar data one by one, or should I do gridding and merge multiple radars in one go? gridding and merging all radars like this:

   grid = pyart.map.grid_from_radars([radar1, radar2, radar3, radar4],(41, 501, 501),
                  ((0.,15000.),(-900000.,900000.),(-900000.,900000.)), 
                                  weighting_function='Barnes2',
                                 fields=['reflectivity', 'velocity'],
                                 gatefilters=gatefilter)

[syedhamidali]

I was able to install and import it using conda create -n dda python=3.10 tensorflow tensorflow-probability jax arm_pyart jupyter -c conda-forge. Thank you for your quick response.

[rcjackson]

Cool. This looks like it’s a Python 3.8 issue. I am fine with dropping support for Python 3.8 in the next release.

From: Hamid Ali Syed @.> Date: Monday, November 21, 2022 at 3:25 PM To: openradar/PyDDA @.> Cc: Jackson, Robert @.>, Mention @.> Subject: Re: [openradar/PyDDA] import and installation error (Issue #76)

I was able to install and import it using conda create -n dda python=3.10 tensorflow tensorflow-probability jax arm_pyart jupyter -c conda-forge. [Screen Shot 2022-11-21 at 4 21 55 PM]https://user-images.githubusercontent.com/35923822/203160780-77f021b2-6b97-43f2-9c55-3bbfeb40baae.png Thank you for your quick response.

— Reply to this email directly, view it on GitHubhttps://github.com/openradar/PyDDA/issues/76#issuecomment-1322665398, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFIQA5HRIFCE5YNNEQADK5TWJPSCVANCNFSM6AAAAAASG6BNL4. You are receiving this because you were mentioned.Message ID: @.***>

[syedhamidali]

And how about this question https://github.com/openradar/PyDDA/issues/76#issuecomment-1322654642 ?

[rcjackson]

Dear Bobby, Thank you for this information. I would also like to know a few more things, as listed below;

1. to perform a multi-doppler analysis, you are using the readily transformed gridded data in the examples. Should I do gridding to radar data one by one, or should I do gridding and merge multiple radars in one go? gridding and merging all radars like this:

grid = pyart.map.grid_from_radars([radar1, radar2, radar3, radar4],(41, 501, 501),
                   ((0.,15000.),(-900000.,900000.),(-900000.,900000.)), 
                                   weighting_function='Barnes2',
                                  fields=['reflectivity', 'velocity'],
                                  gatefilters=gatefilter)

You should grid each radar one by one and make sure that they all have the same grid specification.

[Qxaqq]

grid = pyart.map.grid_from_radars([radar1, radar2, radar3, radar4],(41, 501, 501), ((0.,15000.),(-900000.,900000.),(-900000.,900000.)), weighting_function='Barnes2', fields=['reflectivity', 'velocity'], gatefilters=gatefilter)

@syedhamidali 
Can I ask you how to grid each radar one by one and make sure that they all have the same grid specification?

[syedhamidali]

@Qxaqq Here is the notebook: https://openradarscience.org/ams-open-radar-2023/notebooks/pydda/Retrieving_winds_with_pydda.html

[Qxaqq]

@syedhamidali Thank you.I saw your codeconda create -n dda python=3.10 tensorflow tensorflow-probability jax arm_pyart jupyter -c conda-forge.But i cannot successfully create a python=3.10 environment.I've been using a python=3.9 environment.Can you help me?What is your PyDDA version?What system is your computer?

[syedhamidali]

@Qxaqq I am using MacOS, what is the error exactly?

[Qxaqq]

@rcjackson I am using Windows.Packages are not compatible with each other.Example:tensorflow numpy py-art xarray. UnsatisfiableError:The following specifications were found to be incompatible with each other

@rcjackson @syedhamidali Another question is, what do the black circles and yellow arcs in the plot_horiz of the dual-radar DDA in the example mean?