Possible timeline for the package

[chiaweh2]

Hi aerobulk-python team,

I am not sure this is the right place to post or this would be classified as an "issue". I discovered this great package through the talk at AMS presented by @paigem . It is great to see the originally fortran code can be utilized on Xarray object now. I would love to start using it since my current and past project is highly related to air-sea flux analysis based on reanalysis and simulation output. My question is related to the development timeline of this package. I understand that current guideline would be to use the "aerobulk-python developer guide" listed out in readme. However, I am wondering if there will be any recent official release for general user for this package. If not, would you advice me to just follow the developer guide for now? Thank you and thank you for the great work!

[jbusecke]

Hey @chiaweh2, thanks for reaching out and for the interest in our software!

I think the code mostly works (be aware that the skin still is not threadsafe, so you might run into problems there)! But generally I recommend to just go ahead and try it out! It would be really valuable to see if this works well in a different context. Any feedback/comments on your experience are much appreciated.

To cite this work, it would be easy to create a DOI.

[chiaweh2]

@jbusecke Thank you so much for the response! I just tested the package yesterday and find the installation step very helpful. I also notice one thing that might or might not be a bug. When using the method noskin with default slp (sea level pressure) value, there will be a error code related to AttributeError shown below

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[60], line 1
----> 1 [a,b,c,d,e]=noskin(
      2        da_Tskin_Rechunked,
      3        da_T2m_Rechunked,
      4        da_q2m_Rechunked,
      5        da_uwnd_Rechunked,
      6        da_vwnd_Rechunked,
      7        slp=101000.0,
      8        algo='ecmwf',
      9        zt=2,
     10        zu=10,
     11       niter=6,
     12        input_range_check=False)

File ~/test_aerobulkpython/aerobulk-python/source/aerobulk/flux.py:272, in noskin(sst, t_zt, hum_zt, u_zu, v_zu, slp, algo, zt, zu, niter, input_range_check)
    219 """xarray wrapper for aerobulk without skin correction.
    220 
    221 Warnings
   (...)
    268     evaporation         [mm/s] aka [kg/m^2/s] (usually <0, as ocean loses water!)
    269 """
    270 _check_algo(algo, VALID_ALGOS)
--> 272 sst, t_zt, hum_zt, u_zu, v_zu, slp = xr.broadcast(
    273     sst, t_zt, hum_zt, u_zu, v_zu, slp
    274 )
    275 if input_range_check:
    276     performance_msg = (
    277         "Checking for misaligned nans and values outside of the valid range is performed by default, but reduces performance. \n"
    278         "If you are sure your data is valid you can deactivate these checks by setting `input_range_check=False`"
    279     )

File ~/miniconda3/envs/aerobulk-python-dev/lib/python3.9/site-packages/xarray/core/alignment.py:1049, in broadcast(exclude, *args)
   1047 if exclude is None:
   1048     exclude = set()
-> 1049 args = align(*args, join="outer", copy=False, exclude=exclude)
   1051 dims_map, common_coords = _get_broadcast_dims_map_common_coords(args, exclude)
   1052 result = [_broadcast_helper(arg, exclude, dims_map, common_coords) for arg in args]

File ~/miniconda3/envs/aerobulk-python-dev/lib/python3.9/site-packages/xarray/core/alignment.py:772, in align(join, copy, indexes, exclude, fill_value, *objects)
    576 """
    577 Given any number of Dataset and/or DataArray objects, returns new
    578 objects with aligned indexes and dimension sizes.
   (...)
    762 
    763 """
    764 aligner = Aligner(
    765     objects,
    766     join=join,
   (...)
    770     fill_value=fill_value,
    771 )
--> 772 aligner.align()
    773 return aligner.results

File ~/miniconda3/envs/aerobulk-python-dev/lib/python3.9/site-packages/xarray/core/alignment.py:556, in Aligner.align(self)
    553     self.results = (obj.copy(deep=self.copy),)
    554     return
--> 556 self.find_matching_indexes()
    557 self.find_matching_unindexed_dims()
    558 self.assert_no_index_conflict()

File ~/miniconda3/envs/aerobulk-python-dev/lib/python3.9/site-packages/xarray/core/alignment.py:262, in Aligner.find_matching_indexes(self)
    259 objects_matching_indexes = []
    261 for obj in self.objects:
--> 262     obj_indexes, obj_index_vars = self._normalize_indexes(obj.xindexes)
    263     objects_matching_indexes.append(obj_indexes)
    264     for key, idx in obj_indexes.items():

AttributeError: 'float' object has no attribute 'xindexes'

This error will disappeared if provide a user defined input of slp as Xarray.DataArray. After a quick check on the noskin. It seems to be due to the xr.broadcast() function called in noskin try to broadcast slp (default as scaler) with other DataArray. I assume this can be fixed with an empty DataArray with the same size and coordinate as the sst input but with all grid point value changed to 101000.0? I would be happy to submit a pull request but would also like to first ask the team to see if the bug also happened or it is due to something else messed up on my end. Thank you!

[TomNicholas]

@chiaweh2 thanks for raising this. What version of xarray are you using? This looks like it could be a bug inside xarray relating to ongoing efforts around flexible indexes. If we can reproduce this bug with a pure xarray example then we can raise it there.

[chiaweh2]

@TomNicholas Ah! It could be related to xarray version since I have not kept my xarray updated. My version is

INSTALLED VERSIONS
------------------
commit: None
python: 3.9.7 (default, Sep 16 2021, 13:09:58) 
[GCC 7.5.0]
python-bits: 64
OS: Linux
OS-release: 4.18.0-425.3.1.el8.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.12.1
libnetcdf: 4.8.1

xarray: 0.21.1
pandas: 1.3.5
numpy: 1.21.2
scipy: 1.7.3
netCDF4: 1.5.7
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: 1.5.2
nc_time_axis: 1.4.0
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: 1.3.2
dask: 2022.01.1
distributed: 2022.01.1
matplotlib: 3.5.1
cartopy: 0.20.2
seaborn: None
numbagg: None
fsspec: 2022.01.0
cupy: None
pint: None
sparse: 0.13.0
setuptools: 58.0.4
pip: 21.2.4
conda: None
pytest: None
IPython: 8.0.1
sphinx: None

The error message changed to the following when testing with xr.broadcast(ds.bottomT,100000)(Do you mean a test like this? I might misunderstand what you meant by "reproduce this bug with a pure xarray example")

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [7], in <module>
----> 1 xr.broadcast(ds.bottomT.isel(time=0),100000)

File ~/miniconda3/envs/coast/lib/python3.9/site-packages/xarray/core/alignment.py:768, in broadcast(exclude, *args)
    766 if exclude is None:
    767     exclude = set()
--> 768 args = align(*args, join="outer", copy=False, exclude=exclude)
    770 dims_map, common_coords = _get_broadcast_dims_map_common_coords(args, exclude)
    771 result = [_broadcast_helper(arg, exclude, dims_map, common_coords) for arg in args]

File ~/miniconda3/envs/coast/lib/python3.9/site-packages/xarray/core/alignment.py:286, in align(join, copy, indexes, exclude, fill_value, *objects)
    284 unlabeled_dim_sizes = defaultdict(set)
    285 for obj in objects:
--> 286     for dim in obj.dims:
    287         if dim not in exclude:
    288             all_coords[dim].append(obj.coords[dim])

AttributeError: 'int' object has no attribute 'dims'

[TomNicholas]

@chiaweh2 that's a fairly old version of xarray (6+ months ago). Try it again with the latest version?

[TomNicholas]

Actually no this looks like a real bug in the current version of xarray, let me raise an issue for it!

[TomNicholas]

Raised here for you! https://github.com/pydata/xarray/issues/7459

[chiaweh2]

Thank you for the help!

[chiaweh2]

A quick question, does this mean if the Xarray team fixed the xr.boardcast bug, this bug in Aerobulk-python will only be fixed if user installed the newest version of xarray? (meaning any previous version would not work.) Thanks!

[TomNicholas]

That is how big fixes work, yes :)

On Fri, Jan 20, 2023, 12:45 PM Chia-Wei Hsu @.***> wrote:

A quick question, does this mean if the Xarray team fixed the xr.boardcast bug, this bug in Aerobulk-python will only be fixed if user installed the newest version of xarray? (meaning any previous version would not work.) Thanks!

— Reply to this email directly, view it on GitHub https://github.com/xgcm/aerobulk-python/issues/58#issuecomment-1398735037, or unsubscribe https://github.com/notifications/unsubscribe-auth/AISNPI7B37SKPA3FBTGQ6ZTWTLFL3ANCNFSM6AAAAAAT6HPM4M . You are receiving this because you were mentioned.Message ID: @.***>