inaos / iron-array

2 stars 0 forks source link

Crash evaluating expressions with zarr proxies #591

Closed FrancescAlted closed 2 years ago

FrancescAlted commented 2 years ago

Here it is a simple example reproducing the crash:

import iarray as ia
import zarr
import numpy as np

shape = (20,)
chunks = (10,)
blocks = (5,)
z1 = zarr.open("prova1.zarr", mode="w", shape=shape, chunks=chunks, dtype=np.float32)
z1[:] = np.linspace(0, 1, 20, dtype=z1.dtype).reshape(shape)
z2 = zarr.open("prova2.zarr", mode="w", shape=shape, chunks=chunks, dtype=np.float32)
z2[:] = np.linspace(0, 1, 20, dtype=z2.dtype).reshape(shape)
z3 = zarr.open("prova3.zarr", mode="w", shape=shape, chunks=chunks, dtype=np.float32)
z3[:] = np.linspace(0, 1, 20, dtype=z3.dtype).reshape(shape)

ia1 = ia.zarr_proxy("prova1.zarr", blocks=blocks)
ia2 = ia.zarr_proxy("prova2.zarr", blocks=blocks)
ia3 = ia.zarr_proxy("prova3.zarr", blocks=blocks)
prova_mean_expr = ia.expr_from_string("(p1 + p2 + p3) / 3", {'p1': ia1, 'p2': ia2, 'p3': ia3})
prova_mean = prova_mean_expr.eval()
print(prova_mean)

I am getting this error:

/Users/faltet/miniconda3/envs/iron-array-python/bin/python /Users/faltet/iarray/iron-array-python/examples/prova.py
Fatal Python error: PyThreadState_Get: the function must be called with the GIL held, but the GIL is released (the current Python thread state is NULL)
Python runtime state: initialized

Thread 0x00000001066b4600 (most recent call first):
  File "/Users/faltet/iarray/iron-array-python/iarray/expression.py", line 74 in eval
  File "/Users/faltet/iarray/iron-array-python/examples/prova.py", line 20 in <module>

Process finished with exit code 1
martaiborra commented 2 years ago

Fixed in https://github.com/inaos/iron-array-python/pull/185 and https://github.com/inaos/iron-array/commit/a7110095afe6ced8ccabc316ff3a1d65e5723565