search

pycroscopy / sidpy

Python utilities for storing, processing, and visualizing spectroscopic and imaging data
https://pycroscopy.github.io/sidpy/
MIT License
11 stars 14 forks source link

Explanation of sidpy vs. xarray? #177

Closed nsulmol closed 1 year ago

nsulmol commented 1 year ago

This is likely not the most appropriate location to ask this question, but I could not find another. Please delete if inappropriate.

I (believe I) understand the purpose of sidpy and its use with scifireaders, to abstract away microscopy file formats and allow a single interface to interact with data from multiple devices.

One thing that is less clear to me, however, is the value of using the sidpy base class vs. xarray. It is my understanding that both use dask.array under the hood, but xarray has a larger user base. In principle, scifireaders could output xarray objects, which could be used throughout pycroscopy.

I ask this here because I suppose others who consider using the pycroscopy ecosystem may ask themselves the same question. There could be any valid number of reasons, but explicitly stating them could be useful. If the reason is legacy/historical, and it may make sense to update pycroscopy in the future, that would also be useful to know.

I hope it is clear I have no ill intentions with this question. Thank you for your work developing this great set of tools!

ramav87 commented 1 year ago

It's a long story, but primarily, the reason is because we are largely dealing here with microscopy (spectral imaging) data, and as such designed our data structure to accommodate the particular needs of the microscopy community. Things like point-click visualizers, easy parallelization, scale bars, etc. XArray is more generalized and while powerful, the generality also means that we would have to anyway extend the xarray significantly to incorporate all the things that sidpy can do currently.

nsulmol commented 1 year ago

Hi ramav87,

Alright, thank you for the quick reply and explanation.

nsulmol commented 1 year ago

Closing.

nsulmol commented 10 months ago

@ramav87 : when you get the time, do you think you could expand a bit more on the reasoning of the existing structure vs. xarray? With hindsight, it seems this could have been implemented in xarray, with your own predefined format (obviously, xarray may have been not chosen because we weren't aware of it, or it was missing functionality at the time).

Is sidpy (with scifireaders) still used extensively within Oak Ridge National lab?