search

arthurpessa / ordpy

A Python package for data analysis with permutation entropy and ordinal network methods.
MIT License
81 stars 16 forks source link

Question: how is your package managing nans? #9

Closed paulinesimon97 closed 2 months ago

paulinesimon97 commented 2 months ago

Hi, I would like to use your package in satellite data, automatically. Some data will surely be missing (individually or by patches) and by default replaced by np.nan. Do your package know how to manage nans or do I need to interpolate/replace them? If I need to interpolate/replace them, what do you think is the best way to avoid affecting the final result? Thank you in advance for your help on these questions.

arthurpessa commented 2 months ago

Good afternoon. The package is reliant on the np.sort and np.argsort functions, so the behavior of the functions in ordpy will reflect how these numpy functions behave with np.nan values.

When building the package, we did not have this possibility too much in mind. My guess is that you should probably try different approaches and see the one which suits best your purposes.

On a last note, we have recently developed a generalization of permutation entropy that allows one to handle irregular ("non-grid-like") data, which would be the case of your satellite data with nans as missing values. If it interests you, you can look it up on https://github.com/hvribeiro/knnpe.

Perhaps, more information regarding your data and intentions could help us to tell you something more concrete.