MHKiT-Software / MHKiT-MATLAB

MHKiT-MATLAB provides the marine renewable energy (MRE) community tools for data processing, visualization, quality control, resource assessment, and device performance.
https://mhkit-software.github.io/MHKiT/
BSD 3-Clause "New" or "Revised" License
15 stars 23 forks source link

Add xarray converter #128

Open simmsa opened 2 months ago

simmsa commented 2 months ago

Create a standardized method for converting xarray objects into a matlab structure and converting them back into xarray objects.

simmsa commented 2 months ago

@hivanov-nrel, we should consider this a starting point for xarray conversion. The overall goal is provide a way to use xarray objects created by MHKiT in MHKiT-MATLAB.

Overall there seem to be a few approaches:

  1. Completely convert the xarray object into a MATLAB type
  2. Partially convert the xarray object into a MATLAB type
  3. Don't convert the xarray object and write MATLAB functions to access data

I started with approach 1, attempting to convert xarray to a dictionary, then convert that to json, then decode the json into a struct in MATLAB. This approach is valid, but is going to be suboptimal for a few reasons:

To implement approach 2 xarray has the to_numpy, and to_dict methods that we could leverage. An idea for a matlab struct would be:

The trick with approach 2 is that in some cases we have to alter the data to display it to the MATLAB user. For one dimensional data this is easy, but for multidimensional data we would require user input. This might be confusing for the end user, but it forces them to better understand the underlying data.

Approach 3 could work as well, but may be confusing to MATLAB users. We would probably have to wrap a subset of xarray functionality inside of MATLAB functions. This could expose all of the power of xarray with minimal type conversion.

Approach 2 is presenting itself as a good compromise and probably the preferred path forward, but I'd like to get your feedback.

simmsa commented 1 month ago

@jmcvey3 directed us to these two functions in MHKiT-Python that perform most of the conversion that we need (Thank you!):

save_mat: https://github.com/MHKiT-Software/MHKiT-Python/blob/2ccc286a65685e5e2f5ab68a467209917f0161d9/mhkit/dolfyn/io/api.py#L240

load_mat: https://github.com/MHKiT-Software/MHKiT-Python/blob/2ccc286a65685e5e2f5ab68a467209917f0161d9/mhkit/dolfyn/io/api.py#L309

We may need to make these functions more generic, and be careful about timestamp conversion. In MHKiT-Python we could break out the xarray -> dict conversion, and the dict -> xarray conversion so we can pass a dictionary into MATLAB which can be converted to a struct and pass back a struct for conversion to xarray.

We should probably test xarray here and include the final functions in mhkit_python_utils. If this modified solution meets our needs we can add it to MHKiT-Python.