apache / datafusion-python

Apache DataFusion Python Bindings
https://datafusion.apache.org/python
Apache License 2.0
323 stars 64 forks source link

Enable mimalloc local_dynamic_tls feature #540

Closed jdye64 closed 7 months ago

jdye64 commented 7 months ago

We have been experiencing issues in dask-sql with dynamic TLS allocation when building on linux and targeting aarch64. More details on the errors and approaches can be found at https://github.com/dask-contrib/dask-sql/issues/1169

However, it seems practical to me to at least propose enabling the mimalloc local_dynamic_tls feature by default in ADP so that downstream libraries do you encounter this sort of behavior. I am open to other suggestions or configuration options as well but just wanted to get something started.