Performance difference in executing UDFs with Python and Python-Jep

[JanssenBrm]

I've noticed quite a big performance difference when executing UDFs using the Python and Python-Jep environments. For example:

• j-240326bbc4f54447ad4e120535f7921d was run using the Python-Jep environment and took around 4 hours to process
• j-2403262fa6974750ac4d5da3024c4138 is the same job but with the Python environment and it took just 20 min finish

You can test this yourself with the following script: https://github.com/Open-EO/FuseTS/blob/main/src/fusets/openeo/services/publish_mogpr.py

You just need to use uncomment the execute_udf function and comment out the generate_mogpr_udp function. The runtime is configured in that same script: https://github.com/Open-EO/FuseTS/blob/main/src/fusets/openeo/services/publish_mogpr.py#L77

Unfortunately this makes the Python-Jep runtime not useable in our case. This also means that we cannot use the apply_metadata fixes that were solved in https://github.com/Open-EO/openeo-geotrellis-extensions/issues/266

[jdries]

To clarify, this is about the infamous MOGPR method, which is compute intensive and can become very slow quite easily. The logging is also not optimal, and some things can only be found in yarn logs for the moment.

I think there's two possible strategies:

• checking if Cython and possibly other important dependencies work properly in a jep environment
• checking if the tensorflow based version of GPy is faster: https://gpflow.github.io/GPflow/2.4.0/notebooks/advanced/multioutput.html

I got this from the jep based run:

warning in stationary: failed to import cython module: falling back to numpy
warning in coregionalize: failed to import cython module: falling back to numpy
warning in choleskies: failed to import cython module: falling back to numpy

tmp/venv/GPy/kern/src/symmetric.py:56: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if symmetry_type is 'odd':
tmp/venv/GPy/kern/src/symmetric.py:58: SyntaxWarning: "is" with a literal. Did you mean "=="?
  elif symmetry_type is 'even':
 tmp/venv/GPy/models/mrd.py:307: SyntaxWarning:"is not" with a literal. Did you mean "!="?
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/fromnumeric.py:3474: RuntimeWarning:Mean of empty slice.
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:189: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:264: RuntimeWarning:Degrees of freedom <= 0 for slice
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:222: RuntimeWarning:invalid value encountered in true_divide
 /opt/venv/lib64/python3.8/site-packages/numpy/core/_methods.py:256: RuntimeWarning:invalid value encountered in true_divide
 tmp/venv/GPy/kern/src/stationary.py:168: RuntimeWarning:overflow encountered in true_divide
 tmp/venv/GPy/kern/src/stationary.py:484: RuntimeWarning:overflow encountered in multiply
 tmp/venv/GPy/kern/src/stationary.py:484: RuntimeWarning:invalid value encountered in multiply
 tmp/venv/GPy/kern/src/stationary.py:487: RuntimeWarning:overflow encountered in multiply
 tmp/venv/GPy/kern/src/stationary.py:487: RuntimeWarning:invalid value encountered in multiply