Complex numbers in openEO

[clausmichele]

At Eurac we are working on the improvements of the SAR2Cube project, where we are using openEO to process Sentinel-1 SLC complex data. @aljacob

I create this issue to collect ideas and feedbacks about introducing the possibility to handle complex numbers in openEO.

In my opinion, the first step should be introducing a new process in Math->Constants for the imaginary unit j (or i, depending on how we want to name it). If the complex numbers will be compatible with the already existing implementation depends on the back-ends, but for the Python based ones it will be doable. For sure, a process like absolute should be firstly updated with a mention to the support of complex numbers if we proceed.

[clausmichele]

https://github.com/Open-EO/openeo-processes/pull/456

[clausmichele]

@soxofaan the possibilities will include handling datacubes with complex data and also creation of new datacubes based on real + imag bands (imagine a reduce_dimension over bands where we do real + i*imag).

[clausmichele]

@m-mohr @soxofaan I would like to draft a process that takes in a complex number and returns real and imaginary parts. I see two possibilities:

1. A process called in apply_dimension, so that the result could be stored in two new bands. So if the input band is VV, the result could be VV_real and VV_imag.

2. Or maybe it's better to have two single processes (real and imag) and combine them later? I kind prefer this option, since it's more simple and easy to understand.

[soxofaan]

the first step should be introducing a new process in Math->Constants for the imaginary unit

I'm not sure this is the necessary first step. It implies that complex numbers are a thing already, and as far as I known a JSONSchema number is just a real number. So if you want to define a "process" i, you first have to define a schema for complex numbers so you can use that for process i. After that you also have to add it to the specification of all processes where you want to handle complex numbers (e.g. all math processes at least I guess, array manipulation processes, ... ). Not only induces this a lot of changes in a lot of processes, I'm also afraid this will cause a schism in process specs between back-ends that do and don't want to or can support complex numbers.

An alternative solution is introducing a new type of dimension, e.g. "complex", which acts like a "band" dimension, but has just 2 pre-defined "bands"/labels: "real" and "imag". This approach does not require to rewrite a lot of (math related) processes, and you can just do complex number arithmetic with the existing openEO API and processes.

I'm not that familiar with usage of complex numbers in EO, but how do they solve that in other systems, e.g. netCDF-style data storage, ...?

[m-mohr]

I like the thoughts of @soxofaan. We may even want to create a new dimension type for this instead of using "other" so that data cubes with complex numbers can be "enforced" in schemas, similar to the recent addition of the geometry/vector dimension type.

[clausmichele]

the first step should be introducing a new process in Math->Constants for the imaginary unit

I'm not sure this is the necessary first step. It implies that complex numbers are a thing already, and as far as I known a JSONSchema number is just a real number. So if you want to define a "process" i, you first have to define a schema for complex numbers so you can use that for process i. After that you also have to add it to the specification of all processes where you want to handle complex numbers (e.g. all math processes at least I guess, array manipulation processes, ... ). Not only induces this a lot of changes in a lot of processes, I'm also afraid this will cause a schism in process specs between back-ends that do and don't want to or can support complex numbers.

True, and it's how I used openEO so far. Handling real and imaginary parts as different bands, This is the feedback I was looking for. I will try to proceed as far as I can with this approach. However, I would still need the possibility to save complex results, how to do that?

An alternative solution is introducing a new type of dimension, e.g. "complex", which acts like a "band" dimension, but has just 2 pre-defined "bands"/labels: "real" and "imag". This approach does not require to rewrite a lot of (math related) processes, and you can just do complex number arithmetic with the existing openEO API and processes.

In this case we could have complex specific processes checking if the "complex" dimension exists?

I'm not that familiar with usage of complex numbers in EO, but how do they solve that in other systems, e.g. netCDF-style data storage, ...?

netCDF does not support CFloat data (sse this reply https://stackoverflow.com/a/47171089), geoTIFF does.

[m-mohr]

In this case we could have complex specific processes checking if the "complex" dimension exists?

Yes, and it would also imply that the data type in storage is Complex in case the dimension exists.

[clausmichele]

Ok, so save_result could check if this dimension exists and act accordingly, merging real and imaginary parts in a single output.