[BUG-REPORT] - Issue with using vaex export_csv in a celery / airflow process

[balihoo-gsuri]

@JovanVeljanoski - Facing an Issue with using vaex export_csv in a celery / airflow process.

Description When trying to export a vaex dataframe in a celery worker or an Airflow DAG, an issue occurs ""daemonic processes are not allowed to have children"

Software information

• Vaex version {'vaex': '4.9.1', 'vaex-core': '4.9.2', 'vaex-viz': '0.5.2', 'vaex-hdf5': '0.12.2', 'vaex-server': '0.8.1', 'vaex-astro': '0.9.1', 'vaex-jupyter': '0.7.0', 'vaex-ml': '0.17.0'}
• Vaex was installed via: pip
• OS: Ubuntu 20.10
• celery==5.1.2
• apache-airflow==2.2.2

Any help is appreciated.

[balihoo-gsuri]

Update: The issue was in the use of the apply function of vaex df. Got it working by passing multiprocessing=False to the apply function